User interface including content from a remote device

ABSTRACT

A user interface displayed by an electronic device can include a first user-interface portion generated by the electronic device and a second user interface portion generated by a remote device. The electronic device can detect received input and determine whether the input corresponds to an interactive element in the first user-interface portion or an interactive element in the second user interface portion. If the input corresponds to an interactive element in the first user-interface portion, the electronic device can generate a response to the input. If the input corresponds to an interactive element in the second user-interface portion, the electronic device can transmit information indicative of the input to the remote device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.14/039,557, filed Sep. 27, 2013; which is a continuation of U.S.application Ser. No. 13/175,581, filed Jul. 1, 2011; which is acontinuation-in-part of U.S. application Ser. No. 12/683,218, filed Jan.6, 2010; which is a continuation-in-part of U.S. application Ser. No.12/119,960, filed May 13, 2008; all entitled “Pushing a User Interfaceto a Remote Device,” the disclosures of which are incorporated byreference herein in their entirety.

FIELD

The present invention relates generally to accessories for portablemedia devices and in particular to allowing a portable media device toprovide a user interface for controlling the media device to anaccessory.

BACKGROUND

Portable media devices allow users to store media content (music,videos, photos, audiobooks, etc.) and access stored media content fromnearly anywhere. Some portable media devices also allow users to connectother devices (referred to herein as “accessories” or “accessorydevices”) to the portable media device, thereby enhancing some aspect ofthe portable media device's operation. For example, most portable mediadevices provide a headphone jack for listening to audio content. Someaccessories provide speakers, thus allowing the user to share the soundwith others, or to listen in environments where use of headphones mightnot be feasible (such as while driving).

In some instances, an accessory can be used to control operation of aportable media device; such accessories are referred to herein as“remote control devices.” The remote control device can send commands tothe portable media device indicating a user-requested operation (such asstarting or pausing playback, skipping to the next track, returning to aprevious track, fast-forward or rewind within a track, etc.). Theportable media device can execute the requested operation, thus allowingthe user to operate the portable media device without touching theportable media device. Such remote control operation can be particularlyuseful in situations where it is inconvenient for a user to manipulatethe portable media device's interface directly. For example, portablemedia devices tend to be small, with relatively small controls anddisplay screens. Thus, it can be difficult for a user to operate such adevice while driving or from across a room.

Some remote control devices provide a graphical user interface (GUI) andallow the user to perform more advanced functions such as browsing adatabase of stored content, selecting content to play, etc. Forinstance, Johnson Controls International has been developing a “MobileDevice Gateway” for use in vehicles; the system includes a connectionpoint for a portable media device and a console that provides anaudio/visual interface. But existing remote GUIs are defined andcontrolled by the remote control device, and consequently, they may bearlittle resemblance to a GUI supplied by the portable media deviceitself. Certain functions available on the portable media device (suchas browsing or searching a database, adjusting playback settings, etc.)may be unavailable or difficult to find. Thus, a user may not be able toperform desired functions. Further, GUIs provided for the same portablemedia device by different remote control devices might be quitedifferent, and the user who connects a portable media device todifferent accessories with remote control may find the inconsistenciesfrustrating.

It would, therefore, be desirable to provide a more consistent remoteuser interface experience.

BRIEF SUMMARY

Embodiments of the present invention relate to a graphical userinterface (“GUI”) on an accessory device. The accessory device canprovide a combination of user input and visual feedback devices, such asa video screen for presenting information and feedback to a user, alongwith buttons, knobs, touchscreen and/or touchpad for receiving userinput. The portable media device can provide the accessory with an image(referred to herein as a “remote” GUI image) to be displayed on aportion of the video screen; the image can include various userinterface elements that can resemble or replicate a “native” GUIprovided directly on the portable media device. The accessory can sendinformation to the portable media device indicative of a user actiontaken in response to the displayed image; such information can indicate,for example, that a particular button was pressed or that a particularportion of a touch-sensitive display screen was touched by the user. Theportable media device can process this input to identify the actionrequested by the user and take the appropriate action. The action mayinclude providing to the accessory an updated GUI image to be displayed,where the updated GUI image reflects the user action.

In some embodiments, a user interface displayed by an electronic devicecan include a first user-interface portion generated by the electronicdevice and a second user interface portion generated by a remote device.The electronic device can detect received input and determine whetherthe input corresponds to an interactive element in the firstuser-interface portion or an interactive element in the second userinterface portion. If the input corresponds to an interactive element inthe first user-interface portion, the electronic device can generate aresponse to the input. If the input corresponds to an interactiveelement in the second user-interface portion, the electronic device cantransmit information indicative of the input to the remote device.

The following detailed description together with the accompanyingdrawings will provide a better understanding of the nature andadvantages of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates a portable media device, and FIGS. 1B and 1Cillustrate accessory devices with remote graphical user interfaces forthe portable media device of FIG. 1A according to embodiments of thepresent invention.

FIG. 2 is a block diagram of a system including a portable media deviceand an accessory according to an embodiment of the present invention.

FIG. 3 is a table of commands usable to implement a remote graphicaluser interface (“GUI”) according to an embodiment of the presentinvention.

FIG. 4 is a table of commands usable to implement a remote GUI accordingto another embodiment of the present invention.

FIG. 5 is a flow diagram of a process that can be used to provide aremote GUI for a portable media device on an accessory device accordingto an embodiment of the present invention.

FIG. 6 is a flow diagram of a process that can be used in an accessoryto support a remote GUI according to an embodiment of the presentinvention.

FIG. 7 is a flow diagram of a process that can be used in a portablemedia device to provide a remote GUI to an accessory according to anembodiment of the present invention.

FIG. 8 illustrates a native GUI of a portable media device.

FIGS. 9A and 9B illustrate remote GUI images for an accessory accordingto embodiments of the present invention.

FIGS. 10A-10C illustrate an accessory displaying various types of remoteuser interface images according to an embodiment of the presentinvention.

FIG. 11 is a flow diagram of a process that can be used in an accessoryfor displaying a remote user interface image for a third-partyapplication according to an embodiment of the present invention.

FIG. 12 is a flow diagram of a process that can be used in a portablemedia device for providing a remote user interface image for athird-party application to an accessory according to an embodiment ofthe present invention.

FIG. 13 is a flow diagram of a process that can be used in an accessoryfor displaying a remote user interface image for a third-partyapplication according to another embodiment of the present invention.

FIG. 14 is a flow diagram of a process that can be used in a portablemedia device for providing a remote user interface image for athird-party application to an accessory according to another embodimentof the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention relate to providing a graphicaluser interface (“GUI”) on a remote control accessory device that isdefined and managed by a portable media device rather than the accessorydevice. The accessory device provides a combination of user input andvisual feedback devices, such as a video screen for presentinginformation and feedback to a user, along with buttons, knobs,touchscreen and/or touchpad for receiving user input. The portable mediadevice can provide the accessory with an image to be displayed on thevideo screen; the image can include various user interface elements thatcan resemble or replicate the GUI provided directly on the portablemedia device. The accessory can send information to the portable mediadevice indicative of a user action taken in response to the displayedimage; such information can indicate, for example, that a particularbutton was pressed or that a particular portion of a touch-sensitivedisplay screen was touched by the user. The portable media device canprocess this input to identify the action requested by the user and takethe appropriate action. The action may include providing to theaccessory an updated GUI image to be displayed, where the updated GUIimage reflects the user action.

FIGS. 1A and 1B show portable media device (“PMD”) 102 and accessory 104according to an embodiment of the present invention. In FIG. 1A, PMD 102has a user interface that can include display screen 106 and user inputdevice 108 (e.g., a click wheel as found on certain iPod™ media playersmanufactured and sold by Apple Inc., assignee of the presentapplication). Display screen 106 can present a GUI image to the user,and the user can interact with the GUI image by operating user inputdevice 108. Thus, for example, the user can navigate a database ofstored media content, select one or more media assets to be played,control playback, and adjust settings of PMD 102. The interface providedby display screen 106 and input device 108 is referred to herein as the“native GUI” of PMD 102.

FIG. 1A shows an example of a native GUI image that can be displayed byone embodiment of PMD 102. In this example, native GUI image 101includes an album name (“ALBUM”) at the top and a listing of tracksbeneath. A currently selected track (“Track 5”) is highlighted. Up anddown arrows 105, 107 indicate that scrolling to view additional tracksis possible, and a slider element 109 indicates the approximate positionof the currently selected track within the list of tracks.

PMD 102 also has a connector 110 disposed on its bottom surface 112.Connector 110 provides a mechanical and electrical coupling to otherdevices.

Referring to FIG. 1B, accessory device (also referred to as “accessory”)104 can be an in-vehicle media control unit that can be installed in adashboard of a vehicle such as an automobile, on a vehicle seat back(e.g., in an automobile or airplane), or elsewhere. Accessory 104 caninclude display 114 and buttons 116 a-h arranged near the edges ofdisplay 114. Buttons 116 can provide “soft keys” whose function can beinterpreted by accessory 104 based on what is currently displayed ondisplay 114.

In operation, accessory 104 can play media content. For example,accessory 104 can display video content on display 114 and/or deliverthe video content to another display device (e.g., on the back of adriver or passenger seat). Accessory 104 can deliver audio content tothe vehicle's speaker system. Accessory 104 can also provide control forother functionality; for example, accessory 104 may provideenvironmental controls (heater, air conditioning); navigation-relatedcontrols (interactive maps, driving directions); controls for a radiotuner, DVD player, or the like; and so on.

Accessory 104 can also include a docking bay 118 adapted to receive PMD102. For example, docking bay 118 can be sized and shaped to receive atleast the bottom portion of PMD 102 and can include a connector 120 thatmates with connector 110 of PMD 102, thereby allowing electrical signalsto be transmitted back and forth between PMD 102 and accessory 104. Inan alternative embodiment, PMD 102 and accessory 104 can each beequipped with a wireless communication interface (e.g., an interfaceimplementing Bluetooth standards), thereby allowing the two devices toexchange information without a direct physical connection being made.

In accordance with an embodiment of the present invention, when PMD 102of FIG. 1A is connected to accessory 104 of FIG. 1B, accessory 104 canenter a “remote GUI” operating mode. In this operating mode, PMD 102 canprovide a GUI image to accessory 104. Accessory 104 can display theimage on display 114. This remote GUI image can be displayed asreceived, without modification by accessory 104, thereby allowing PMD102 to control the look of the interface. In some embodiments, PMD 102provides a remote GUI image that replicates the native GUI of PMD 102.In other embodiments, PMD 102 can provide a remote GUI image thatconforms to a modified version of the native GUI. For example, themodified GUI may use different fonts or color schemes, or may includeadditional or different control options.

In FIG. 1B, display 114 of accessory 104 displays an example remote GUIimage 111. Like native GUI image 101 shown in FIG. 1A, remote GUI image111 includes an album name (“ALBUM”) at the top and a listing of tracksbeneath. A currently selected track (“Track 5”) is highlighted. Up anddown arrows 113, 115 indicate that scrolling to view additional tracksis possible, and a slider element 117 indicates the approximate positionof the currently selected track within the list of tracks. Up arrow 113and down arrow 115 are placed adjacent to buttons 116 e and 116 h,respectively, thereby indicating that button 116 e can be used to moveup the list while button 116 h can be used to move down the list. Asdescribed below, the mapping of operations to buttons 116 a-h can bedetermined by PMD 102. When a user presses one of buttons 116 a-h,accessory 104 can send to PMD 102 a signal indicating which of buttons116 a-h was pressed. PMD 102 can interpret the signal and takeappropriate action. The action may include updating the remote GUIimage, starting or pausing playback, modifying a setting or instructingaccessory 104 to modify a setting, etc.

Remote GUI image 111 on display 114 of accessory 104 can also includeadditional elements not found on native GUI image 101. For example,remote GUI image 111 includes GUI elements indicating a “Play” operation(element 119) and an “Exit” operation (element 121). Elements 119 and121 are positioned next to buttons 116 a and 116 d, respectively, toindicate that the corresponding operations can be invoked by pressingthe adjacent button. In this example, the user can press button 116 a toindicate that the currently selected track should be played. The usercan press button 116 d to indicate that accessory 104 should exit theremote GUI mode, allowing the user to access other functionality (e.g.,navigation, climate control, etc.). Interfaces for such otherfunctionality might or might not also be controlled by PMD 102. In someembodiments, exiting the remote GUI mode does not disconnect accessory104 from PMD 102; for example, PMD 102 and accessory 104 can continueplaying a media track while the other functionality of accessory 104 isaccessed.

FIG. 1C illustrates accessory 150 according to another embodiment of thepresent invention. Like accessory 104, accessory 150 can be anin-vehicle media control unit. Accessory 150 includes display 152 anddocking bay 154 adapted to receive a PMD such as PMD 102 of FIG. 1A. Inthis example, docking bay 154 includes a connector 156 that mates withconnector 110 of PMD 102, thereby allowing electrical signals to betransmitted back and forth between PMD 102 and accessory 150. In analternative embodiment, PMD 102 and accessory 150 can each be equippedwith a wireless communication interface (e.g., an interface implementingBluetooth standards), thereby allowing the two devices to exchangeinformation without a direct physical connection being made.

Display 152 of accessory 150 has a touch-sensitive overlay such thatwhen a user touches a portion of display 152, the touched portion can beidentified, e.g., using (x, y) pixel coordinates. A user can touch anelement of the displayed GUI image to select an action to be taken, assuggested by hand 158. In other embodiments, the touch-sensitive overlayof display 152 can also detect finger motions such as dragging along thesurface, or opening or closing fingers.

Like accessory 104 of FIG. 1B, when PMD 102 is connected to accessory150, accessory 150 can enter a remote GUI operating mode and can displaya remote GUI image 151 provided by PMD 102. Like remote GUI image 111 inFIG. 1B, remote GUI image 151 can be a modified version of native GUIimage 101 of FIG. 1A. Remote GUI image 151 includes an album name(“ALBUM”) at the top and a listing of tracks beneath. A currentlyselected track (“Track 5”) is highlighted. Up and down arrows 153, 155indicate that scrolling to view additional tracks is possible, and aslider element 157 indicates the approximate position of the currentlyselected track within the list of tracks. In this instance, the user cantouch an area of the screen to select the corresponding action. Thus,for example, as shown in FIG. 1C, the user (hand 158) can touch downarrow 155 to advance the selection through the list (e.g., from “Track5” to “Track 6”).

Remote GUI image 151 can also include additional elements not found onnative GUI image 101. For example, remote GUI image 151 includes GUIelements indicating a “Play” operation (element 159), a “Back” operation(element 161), and an “Exit” operation (element 163). In this example,the user can touch “Play” element 159 to indicate that the currentlyselected track should be played. The user can touch “Back” element 161to navigate backward in the database hierarchy (e.g., to a list ofalbums from which the currently displayed album was selected). The usercan touch “Exit” element 163 to indicate that accessory 150 should exitthe remote GUI mode, allowing the user to access other functionality(e.g., navigation, climate control, etc.). As noted above, interfacesfor such other functionality might not be controlled by PMD 102. Also asnoted above, exiting the remote GUI mode need not disconnect accessory150 from PMD 102; for example, PMD 102 and accessory 150 can continueplaying a media track while the other functionality is accessed.

As described below, the mapping of GUI elements to screen areas can bedetermined by PMD 102. When a user touches a portion of display screen152, accessory 154 can simply send to PMD 102 a signal indicating whichportion of the screen was touched (e.g., pixel coordinates of thetouched location). PMD 102 can interpret the signal and take appropriateaction. The appropriate action may include updating the remote GUIimage, starting or pausing playback, modifying a setting or instructingaccessory 104 to modify a setting, etc.

At different times, PMD 102 of FIG. 1A can be coupled to differentaccessories. Thus, for example, at one time PMD 102 can be coupled toaccessory 104 of FIG. 1B and can provide a remote GUI suitable for usewith an interface where there are buttons 116 a-h. At a different time,PMD 102 can be coupled to accessory 150 of FIG. 1C and can provide aremote GUI suitable for use with a touchscreen interface. Because theremote GUI is under the control of PMD 102, a user can interact with PMD102 directly or through various accessories and experience a generallyconsistent interface from one configuration to the next. Suchconsistency can help to make the use of different interfaces, as well asthe transition from one accessory to another, more intuitive for theuser.

It will be appreciated that the systems of FIGS. 1A-1C are illustrativeand that variations and modifications are possible. A variety ofportable media devices may be used, not limited to PMD 102, and thenative GUI may vary from one device to the next. For instance, some PMDsmay provide touch-screen interfaces, graphic representations of contentlistings (e.g., incorporating images of album covers), animated GUIimages, or other features not specifically illustrated herein. Some PMDsmay also incorporate other functionality in addition to media assetstorage, search, and playback. Examples include personal informationmanagement (e.g., calendar, contacts); telephony (e.g., via mobile phonenetwork); Internet connectivity (e.g., via wireless communicationprotocols such as the IEEE 802.11 family of standards (also known as“WiFi”) and/or wireless data networks such as 3G or EDGE); maps andnavigation; execution of application programs (including third-partyapplication programs loaded onto the PMD by a user); and so on. The PMDmay provide remote GUI interactivity for any or all of its functions,and the combination of functions available may vary depending on theaccessory. (For example, to avoid driver distraction, it might bedesirable to disable Internet browsing or video playback in a movingautomobile.)

A remote GUI mode can be provided in a variety of accessories. Forexample, in addition to in-vehicle systems, a PMD interface with remoteGUI can be provided in a console on an exercise machine (such as atreadmill, stationary bicycle, or the like), in an in-flightentertainment console of a commercial or private airplane, in a homeentertainment system incorporating a video device such as a television(e.g., using a set top box or an integrated component of the videodisplay device to provide an interface to the PMD), and so on.

FIG. 2 is a block diagram of system 200 according to an embodiment ofthe present invention. System 200 can include PMD 202 (e.g.,implementing PMD 102 of FIG. 1A) and accessory 220 (e.g., implementingaccessory 104 of FIG. 1B or accessory 150 of FIG. 1C).

PMD 202 in this embodiment can provide media player capability. PMD 202can include processor 204, storage device 206, user interface 208,network interface 210 and accessory input/output (I/O) interface 214.Processor 204 in this embodiment can be a programmable processor thatexecutes programs to implement operations such as playback of mediatracks and browsing a database of stored media assets, as well as anative GUI to provide user control over the implemented operations.Processor 204 can also implement a remote GUI program that generatesremote GUI images for accessory 220 and processes user input forwardedby accessory 220, as described below.

Storage device 206 may be implemented, e.g., using memory such as disk,flash memory, or any other non-volatile storage medium. In someembodiments, storage device 206 can store a database of media assets(also referred to herein as “tracks”), such as audio, video, stillimages, or the like, that can be played by PMD 202, together withmetadata descriptive of each track. Metadata can include, e.g., a mediatype (audio track, video track, audio book, still image, etc.); an assettitle; a name of an artist or performer associated with the asset;composer or author information; asset length; chapter information; albuminformation; lyrics; information about associated artwork or images;description of the asset; and so on. Other information, includingprograms to be executed by processor 204, can be stored in storagedevice 206.

User interface 208 may include one or more input controls 207 such as atouch pad, touch screen, scroll wheel, click wheel, dial, button,keypad, microphone, or the like, as well as display screen 209. Othercomponents, such as an audio output section (not explicitly shown) canalso be included. A user can view native GUI images generated byprocessor 204 on display screen 209 and can operate input controls 207based on the displayed image to invoke the functionality of PMD 202.Processor 204 can process the user input and take appropriate action,including updating the native GUI image on display screen 209. Thus,user interface 208 and processor 204 can provide a native GUI for PMD202.

Network interface 210 can be operated under control of processor 204 toconnect PMD 202 to various communication networks, including voice anddata networks. For example, network interface 210 can provide local areanetwork connections via WiFi, mobile telephony via suitable RFcircuitry, access to wireless data networks including 3G and EDGEnetworks, personal area connections via Bluetooth, or the like. In someembodiments, network interface 210 can provide wired connections inaddition to or instead of wireless connections.

Accessory I/O interface 214 can allow PMD 202 to communicate withvarious accessories. For example, accessory I/O interface 214 cansupport connections to such accessories as an in-vehicle media system orthe like. In one embodiment, accessory I/O interface 214 includes a30-pin connector corresponding to the connector used on iPod™ productsmanufactured and sold by Apple Inc. or one or more other connectors,such as a Universal Serial Bus (“USB”) or FireWire connector.Alternatively or additionally, accessory I/O interface 214 can include awireless interface (e.g., Bluetooth or the like). Accessory I/Ointerface 214 can allow PMD 202 to communicate with accessory 220 oranother accessory.

Accessory 220 includes controller 224, one or more input controls 222,display 232, PMD I/O interface 226, environmental and status inputinterface 228, and audio output section 230. Controller 224 can include,e.g., a microprocessor or microcontroller executing program code toperform various functions such as digital audio decoding, analog ordigital audio and/or video processing, and the like. Input controls 222can include, e.g., a touch pad, touch screen, scroll wheel, click wheel,dial, button, keypad, microphone, or the like. Display 232 can be usedto present operational or control information as well as video contentto the user; in some embodiments, display 232 can be implemented as amulti-screen display system, and the images shown on different screensmight be the same or different. Thus, for example, one screen can beused for operational or control information while another screen is usedfor presenting video content. In still other embodiments, operational orcontrol information can be overlaid or composed with video content,allowing a user to view both simultaneously on the same screen.

Accessory 220 can be operable in a “local GUI” mode as well as a remoteGUI mode. In the local GUI mode, controller 224 can generate GUI imagesto be displayed on display 232 and can receive and process user inputfrom input controls 222, thereby allowing a user to control variousoperations of accessory 220. In the remote GUI mode, GUI images can begenerated by processor 204 of PMD 202, delivered via accessory I/Ointerface 214 and PMD I/O interface 226 to controller 224, and displayedon display 232. Controller 224 can detect user operation of inputcontrols 222 and send corresponding signals to PMD 202 via PMD I/Ointerface 226 and accessory I/O interface 214. Processor 204 of PMD 202can process the signals to determine what action the user has requested;depending on the user request, processor 204 can generate an updatedremote GUI image to be displayed by display 232, generate otherinstructions to controller 224, or invoke other operations of PMD 202(such as beginning or ending playback, searching the database of storedassets, etc.).

Audio output device 230, which can be implemented, e.g., as one or moreintegrated circuits, provides the capability to output audio. Forexample, audio output device 230 can include one or more speakers ordriver circuits and connectors for external speakers, thereby enablingaudio to be presented to a user. In one embodiment, controller 224 canreceive audio signals from PMD 202 via PMD I/O interface 226 and canprovide the signals with or without further processing to audio outputdevice 230; audio output device 230 can transform the signals asappropriate for presentation to the user.

Environmental and status input interface 228 can include communicationpathways to other systems of the equipment in which accessory 220 isinstalled, allowing these systems to provide accessory 220 withinformation about the operating environment and/or status. For example,if accessory 220 is installed in an automobile, environmental statusinput interface 228 may receive information indicating whether theautomobile's headlights are on or off, what gear the vehicle iscurrently in, whether a parking brake is engaged, current vehicle speed,etc. If accessory 220 is installed in an exercise machine, environmentalstatus input interface 228 may receive information indicating whetherthe machine is in use and information about the current status of aworkout when one is in progress. Accessory 220 can provide thisinformation to PMD 202, and PMD 202 can use the information to customizethe remote GUI images and functionality to the operating environment, asdescribed below.

Accessory 220 can be any accessory that provides a display and one ormore associated user input controls. Examples include in-vehicle mediaunits that can be mounted, e.g., in a dashboard or seat back, consolesthat may be provided on exercise equipment, airplane in-flightentertainment systems (e.g., mounted in a seatback, armrest, or consoleunit), home entertainment systems, and so on. In one embodiment, PMD I/Ointerface 226 includes a 30-pin connector that mates with the connectorused on iPod™ products manufactured and sold by Apple Inc. PMD I/Ointerface 226 can also include other types of connectors, e.g.,Universal Serial Bus (USB) or FireWire connectors. Alternatively, PMDI/O interface 226 can include a wireless interface (e.g., Bluetooth orthe like).

It will be appreciated that the system configurations and componentsdescribed herein are illustrative and that variations and modificationsare possible. The PMD and/or accessory may have other capabilities notspecifically described herein. For example, the PMD can include anycombination of computing and/or communication functionalities inaddition to media playback capability. Examples of other functionalitiesthat can be provided by a PMD include personal information management(e.g., calendar, contacts); telephony (e.g., via mobile phone network);Internet connectivity (e.g., via wireless communication protocols suchas WiFi and/or wireless data networks such as 3G or EDGE); maps andnavigation (e.g., using a Global Positioning System (“GPS”) receiver;execution of application programs (including third-party applicationprograms that may be loaded onto the PMD by a user); and so on.

Further, while the PMD and accessory are described herein with referenceto particular blocks, it is to be understood that these blocks aredefined for convenience of description and are not intended to imply aparticular physical arrangement of component parts. Further, the blocksneed not correspond to physically distinct components. Embodiments ofthe present invention can be realized in a variety of devices includingelectronic devices implemented using any combination of circuitry andsoftware.

Accessory I/O interface 214 of PMD 202 and PMD I/O interface 226 ofaccessory 220 allow PMD 202 to be connected to accessory 220 andsubsequently disconnected from accessory 220. As used herein, PMD 202and accessory 220 are “connected” whenever a communication channelbetween accessory I/O interface 214 and PMD I/O interface 226 is openand are “disconnected” whenever the communication channel is closed.Connection can be achieved by physical attachment (e.g., betweenrespective mating connectors of PMD 202 and accessory 220), by anindirect connection such as a cable, or by establishing a wirelesscommunication channel. Similarly, disconnection can be achieved byphysical detachment, disconnecting a cable, powering down accessory 220or PMD 202, or closing the wireless communication channel. Thus, avariety of communication channels may be used, including wired channelssuch as USB, FireWire, or universal asynchronous receiver/transmitter(“UART”), or wireless channels such as Bluetooth.

Regardless of the particular communication channel, as long as PMD 202and accessory 220 are connected to each other, the devices cancommunicate by exchanging commands and data according to a protocol. Theprotocol defines a format for sending messages between PMD 202 andaccessory 220. For instance, the protocol may specify that each messageis sent in a packet with a header and an optional payload. The headerprovides basic information (e.g., a start indicator, length of thepacket, and a command to be processed by the recipient), while thepayload provides any data associated with the command; the amount ofassociated data can be different for different commands, and somecommands may provide for variable-length payloads. In some embodiments,the commands may be defined such that a particular command is valid inonly one direction. The packet can also include error-detection orerror-correction codes as known in the art.

The protocol can define a number of “lingoes,” where a “lingo” is agroup of related commands that can be supported (or unsupported) byvarious classes of accessories. In one embodiment, a command can beuniquely identified by a first byte identifying the lingo to which thecommand belongs and a second byte identifying the particular commandwithin the lingo. Other command structures may also be used. It is notrequired that all accessories, or all PMDs to which an accessory can beconnected, support every lingo defined within the protocol.

In some embodiments, every accessory 220 and every PMD 202 that aredesigned to be interoperable with each other support at least a“general” lingo that includes commands common to all such devices. Thegeneral lingo can include commands enabling the PMD and the accessory toidentify and authenticate themselves to each other and to providegeneral information about their respective capabilities, including which(if any) other lingoes each supports. The general lingo can also includeauthentication commands that the PMD can use to verify the purportedidentity and capabilities of the accessory (or vice versa), and theaccessory (or PMD) may be blocked from invoking certain commands orlingoes if the authentication is unsuccessful.

A command protocol supported by PMD 202 and accessory 220 can include a“remote GUI” lingo (or other group of commands) that can be used tocommunicate commands and data related to permitting a user to controlthe operation of PMD 202 via a remote GUI provided on accessory 220. Theremote GUI lingo can include commands that accessory 220 can send to PMD202 to provide information about the configuration of display 232 andinput controls 222, as well as environmental information (such as whereaccessory 220 is installed, operational status of a vehicle or otherequipment in or on which accessory 220 is installed, etc.). The remoteGUI lingo can also include commands used by PMD 202 to deliver GUI imagedata to accessory 220 and commands used by accessory 220 to forward userinput signals to PMD 202. Examples are described below.

Commands that can be used to implement a remote GUI according to anembodiment of the invention will now be described. In one embodiment,the commands can include commands usable to provide a PMD withinformation about the configuration of an accessory's display device anduser input control(s). The commands can also include commands usable toprovide the PMD with information about the environment in which theaccessory operates and any special requirements of the accessory, suchas logos or other content to be included in the remote GUI image. Thecommands can also include a command used to send remote GUI image datato the accessory and a command used to send user input information tothe PMD.

FIG. 3 is a table 300 listing commands that can be used to implement aremote GUI according to an embodiment of the present invention. For eachcommand, the direction and payload are indicated. “Direction” refers towhether the command is sent from the PMD to the accessory (denoted“P→A”) or vice versa (denoted “A→P”). “Payload” refers to any data orother parameters associated with the command.

In some embodiments, a SetDisplayInfo command can be sent by accessory220 to PMD 202 to provide information about display 232. In oneembodiment, the information can include any or all of: the displaydimensions (e.g., in pixels or inches or both) of a display device,color depth information for the display device (e.g., whether thedisplay is color or black and white, the number of distinct color valuessupported, etc.); the display format of the display device (e.g., analogor digital input and signal formats); and the refresh rate of thedisplay device. It is to be understood that any other information usefulto configuring an image may be provided to PMD 202. In some embodiments,accessory 220 may provide multiple display devices, and the informationprovided to PMD 202 may include the number of display devices as well asseparate configuration information for each display device.

In some embodiments, a SetControlInfo command can be sent by accessory220 to PMD 202 to provide information about the type, number andlocation of user input controls 222. In one embodiment, the command canfirst identify the number of controls. Then, for each control, thecommand can identify the type (e.g., button, knob, touchscreen) andapproximate location of the control.

In some embodiments, a SetEnvinfo command can be sent by accessory 220to PMD 202 to provide information about the environment in whichaccessory 220 operates. For instance, the environmental information caninclude information as to where accessory 220 is installed, e.g.,vehicle dashboard, airplane in-flight entertainment system; exerciseequipment console, home entertainment system, etc.

In some embodiments, a SetEnvImage command can be sent by accessory 220to PMD 202 to provide an image element that can be included in a remoteGUI image. For example, the image element might be a logo associatedwith the manufacturer or provider of accessory 220 (or of a vehicle orother equipment in which accessory 220 is installed). As describedbelow, PMD 202 can reserve an area of the remote GUI image fordisplaying an accessory-supplied image.

In some embodiments, a SetStatusInfo command can be sent by accessory220 to PMD 202 to provide information about the status of the operatingenvironment. For example, if accessory 220 is installed in an automobiledashboard, status information may include information such as whetherthe automobile is in motion (or in gear); whether it is day or night(e.g., whether the automobile's headlights are off or on); whethernavigational route guidance is in progress; etc. If accessory 220 isinstalled in an exercise machine such as a treadmill, status informationmay include information such as whether the machine is currently in useas well as workout data such as elapsed or remaining time, caloriesburned, heart rate, current speed, or the like. As described below,status information provided by the accessory can be used by PMD 202 toaugment the remote GUI images with status information, to adjust theappearance of the remote GUI images based on the current status, and/orto limit access to certain features (such as video playback) in astatus-dependent manner.

In some embodiments, a RemoteGUIImageData command can be sent by PMD 202to accessory 220 to deliver pixel data for a remote GUI image toaccessory 220. The command can include pixel data for a portion or allof display 232. In some embodiments, a command packet may have a maximumsize, which can limit the amount of pixel data that can be sent with asingle RemoteGUIImageData command. In this case, PMD 202 can usemultiple RemoteGUIImageData commands to send the pixel data; eachcommand may include a parameter identifying the portion of the displayto which the pixel data applies. Any format suitable for sending pixeldata, including compressed formats, can be used. In other embodiments,accessory I/O interface 214 of PMD 202 can incorporate a video outputinterface, which can be analog or digital, and the remote GUI image canbe delivered to accessory 220 through the video output interface ratherthan by sending commands.

In some embodiments a ProcessUserInput command can be sent by accessory220 to PMD 202 to indicate the detection of user input in response to aremote GUI image. The command can include data indicating, e.g., whichuser control was operated and, if applicable, what operation wasperformed. For instance, for accessory 104 of FIG. 1B, theProcessUserInput data can indicate which one of buttons 116 a-h waspressed. For accessory 150 of FIG. 1C, the ProcessUserInput data canindicate pixel coordinates of a location on the screen that was touchedby the user. For a user input device that can be manipulated in multipleways (e.g., turning a knob either clockwise or counterclockwise ordragging a finger across a suitably configured touchscreen), theProcessUserInput data can also indicate which manipulation of the inputdevice was detected.

In some embodiments, an EnterRemoteGUIMode command can be sent fromaccessory 220 to PMD 202 (or vice versa) to initiate remote GUI modeoperation, and an ExitRemoteGUIMode command can be sent from PMD 202 toaccessory 220 (or vice versa) to terminate remote GUI mode operation.

FIG. 4 is a table 400 listing commands that can be used to implement aremote GUI according to another embodiment of the present invention. Asin table 300, for each command the direction and payload are indicated.

In some embodiments, an AccessorySettings command can be sent byaccessory 220 to PMD 202 to provide information about itscharacteristics. The AccessorySettings command can be a general-lingocommand for providing information about the accessory's capabilities,preferences, and/or settings; in some embodiments, this command is alsousable to send information unrelated to remote GUI operations. Thepayload of the AccessorySettings command can include a token identifyingthe type of information being provided (e.g., information about displaycapabilities, information about audio capabilities, etc.) and adescriptor associated with the token that carries the specificinformation. For example, in the case of a display token, the associateddisplay descriptor can include information indicating properties of theaccessory's display screen. In one embodiment, properties can includethe vertical and horizontal physical screen size (e.g., in inches orcentimeters); the vertical and horizontal screen resolution (e.g., innumber of pixels); the vertical and horizontal dimensions of the portionof the screen allocated to the remote GUI (e.g., in number of pixels);the screen's color depth (e.g., in number of bits or simply an indicatorof whether the screen is color or black and white); the screen's gamma(color correction) factor; and other features such as whether thedisplay is touch-sensitive.

In some embodiments, the display descriptor can also include otherinformation, such as the video signal format used by the accessory(e.g., NTSC, PAL, etc.); video connection type (e.g., composite video,S-Video, component video, etc.); video scan mode (e.g., interlaced orprogressive); screen aspect ratio (e.g., 4:3 full-screen or 16:9wide-screen); and so on. In other embodiments, some or all of thisadditional display information can be provided using, e.g., aSetVideoPreferences command of the general lingo. TheSetVideoPreferences command may also provide the ability to adjustsettings related to playback of video content but unrelated to theremote GUI, such as whether to show or hide captions and subtitles whenplaying videos, or whether the video image should be stretched to fillthe screen or leave black bands. (As described above, the dimensions ofthe remote GUI image can be specified using the AccessorySettingscommand and may be unaffected by whether image stretching is selectedfor video playback.)

Those skilled in the art will appreciate that some combinations ofpreferences may cause errors; for example, selection between progressiveand interlaced scan modes may not be permitted for composite video orS-video connections (which support only interlaced scan mode). If anerror occurs, the PMD may send an error message as a command to theaccessory.

In some embodiments, the AccessorySettings command can also be used toprovide information about user input controls that are to be associatedwith the remote GUI. For example, a token associated with input controlinformation can be defined. The associated descriptor can containinformation about the controls, such as the number of controls, the typeof each control (e.g., rotational, button, or touchscreen), theapproximate location of each control, etc., similarly to theSetControlInfo command described above. The accessory can also assigneach control a unique identifier and include the assigned identifier inthe control descriptor. They accessory can subsequently use the assignedidentifier to specify to the PMD which control was operated.

In some embodiments, PMD 202 may be able to selectively expose variouscontent or functionality to accessory 220 via the remote GUI. Forexample, as described above PMD 202 can support multiple functionalitiessuch as audio playback, video playback, still image browsing andplayback, mobile telephony, e-mail, web browser, personal informationstorage and retrieval, execution of application programs (such asthird-party application programs) that may be stored in storage device206, GPS navigation capability, and so on. Further, a particular PMD 202can be part of a family of media and communication devices, each ofwhich may support different combinations of functionalities. In someembodiments, one PMD model might provide only audio playback while asecond model provides audio and video playback, a third model providesaudio and video playback as well as mobile telephony, and so on. Thus,which functionalities are available may vary from one PMD to the next,and which of the available functionalities should be exposed via theremote GUI may depend in part on the accessory and its operatingenvironment.

To allow the accessory to control which of the available functionalitiesare or are not exposed via the remote GUI, some embodiments provide“options” commands, such as the GetPMDOptions, RetPMDOptions, andSetPMDOptions commands of table 400. The GetPMDOptions command can besent by accessory 220 to PMD 202 to request information about whichfunctionalities are available from PMD 202. The payload can include aselector, allowing the accessory to indicate whether the PMD shouldrespond by listing all available functionalities (i.e., allfunctionalities that the PMD supports and is capable of exposing via theremote GUI) or only those functionalities that are active (i.e.,currently selected for exposure via the remote GUI).

The RetPMDOptions command can be sent by PMD 202 to accessory 220 inresponse to the GetPMDOptions command. The payload can provide a list ofavailable or active functionalities, depending on the selector in theGetPMDOptions command. In one embodiment, the list can be implemented asa bitmask with a different bit assigned to each functionality that couldbe exposed via the remote GUI if a particular PMD supports it. Forinstance, bits can be assigned to audio playback, video playback, stillimage browsing, web browsing, GPS navigation, telephone, and so on. Ifthe GetPMDOptions selector requests a list of available functionalities,PMD 202 can set the bits to indicate which functionalities areavailable. If the GetPMDOptions selector requests a list of activefunctionalities, PMD 202 can set the bits to indicate whichfunctionalities are currently active. In other embodiments, the list canbe implemented using other data structures. For example, a unique bytecode can be associated with each functionality, and the payload of theRetPMDOptions command can include a list of byte codes identifyingavailable or active functionalities.

The SetPMDOptions command can be sent by accessory 220 to PMD 202 tospecify the functionalities that should be made active. The payload canbe a bitmask, e.g., using the same mapping of bits to functionalities asthe RetPMDOptions command, and accessory 220 can set the bits accordingto whether a particular functionality should or should not be active.Thus, for example, if accessory 220 is installed in a vehicle, it may bedesirable to expose only functionalities that will not unduly distractthe driver, such as audio but not video playback, or navigation but notweb browsing. Accessory 220 can set the bitmask appropriately for thedesired behavior. Alternatively, in embodiments where byte codes areassociated with each functionality, accessory 220 can send a list of thebyte codes associated with the functionalities that should be active. Insome embodiments, if accessory 220 requests a functionality that aparticular PMD 202 does not support or cannot expose via the remote GUI,PMD 202 can respond with an error message.

PMD 202 can use the payload of the SetPMDOptions command in generatingremote GUI images. For example, menus presented in a remote GUI imagemay include only items associated with active functionalities, or itemsassociated with inactive functionalities may appear in darkened,grayed-out or other de-emphasized form, indicating to the user that theyare not available. In some embodiments, if accessory 220 does not send aSetPMDOptions command, PMD 202 can expose a default set offunctionalities. For example, all available functionalities can beexposed by default, or the default set can be limited to a minimal setof functionalities (e.g., only audio playback).

In some embodiments, accessory 220 can send the SetPMDOptions command atany time, including during remote GUI operation, to dynamically changethe set of active functionalities. For instance, if accessory 220 isinstalled in an automobile, video playback or web browsing functionalitymight be inactive while the automobile is being driven and active whilethe automobile is parked. Whether an automobile is parked can bedetected, e.g., by detecting whether an automatic transmission isshifted into “Park” or whether the automobile's parking brake isengaged. In response to detecting a transition into or out of a parkedstate, accessory 220 can send a new SetPMDOptions command to change theset of active functionalities.

The BrightnessAdjust command can be sent by accessory 220 to PMD 202 tosignal a change in display brightness. The payload can be an indicatorof the desired brightness. In one embodiment, the payload corresponds toa gain setting for a digital-to-analog converter used by PMD 202 togenerate analog video output signals; other indicators of brightness canalso be used. The BrightnessAdjust command can be used, e.g., invehicles that have panel brightness controls, allowing PMD 202 to adjustthe brightness of the remote GUI image in coordination with adjustmentsto brightness of other lights on the panel. Other commands related toadjusting display parameters can also be provided.

The SetStatusInfo command can be similar to the SetStatusInfo commanddescribed above with reference to FIG. 3. This command can be sent byaccessory 220 to PMD 202 to provide information about changes in thestatus of the operating environment, and any of the status informationdescribed above can be provided.

In some embodiments, additional status changes can be provided using theSetStatusInfo command. For example, accessory 220 can be implemented ina head unit (e.g., in an automobile) that is capable of receiving videoand/or audio input from different sources. Video input can be received,from PMD 220 or from a different source such as a built-in GPSnavigation unit or DVD player. Similarly, audio input can be receivedfrom PMD 220 or from a different source such as a built-in radioreceiver or CD or DVD player. A user may be able to operate a control ofaccessory 220 to change the audio and/or video source at any time,regardless of whether PMD 202 is currently providing a remote GUI.Accordingly, the SetStatusInfo command can be used to signal when theaccessory's audio source selection changes to or from PMD 202 and/orwhen the accessory's video source selection changes to or from PMD 202.Depending on implementation, a particular PMD 202 might or might not acton the status-change information provided by the accessory; however, theaccessory can always send the SetStatusInfo command in response to anystatus-change event regardless of whether or how a particular PMD 202might respond.

In the embodiment described above with reference to FIG. 3, accessory220 can communicate any type of user input to PMD 202 using a singleProcessUserInput command. Alternatively, as shown in FIG. 4, differentcommands can be associated with different types of user input controls.For example, depending on the type of input device that is operated,accessory 220 can send a ButtonInput command, a RotationInput command,or a TouchInput command. In some embodiments, accessory 220 can continueto send the appropriate input command at regular intervals (e.g., 20 ms,50 ms, or the like) for as long as the user continues to operate aparticular control.

The ButtonInput command can be used to communicate information about theoperation of a control button to PMD 202. In some embodiments, asdescribed above, accessory 220 can provide identifying information abouteach button in advance, e.g., using the AccessorySettings orSetControlInfo command as described above. In such embodiments, theButtonInput command can simply include an identifier of the button thatis activated, or a bitmask indicating which of the buttons are activatedat the time of the ButtonStatus command. As noted above, the ButtonInputcommand can be sent at regular intervals for as long as a user continuesto hold one or more buttons down. When all buttons are released, a finalButtonStatus command can be sent with a payload indicating that allbuttons have been released.

In other embodiments, accessory 220 is not required to provideinformation about its input controls in advance of use. Instead, theaccessory can internally map its controls to a set of generic GUInavigation buttons, e.g., Up, Down, Left, Right, Select, and Menu. Inthis case, the ButtonStatus command can be implemented with a“control-source” parameter that identifies the location of the activebutton (or other control) and a bitmask associated with the generic GUInavigation buttons. For instance, in one embodiment where accessory 220is an automobile head unit, the control-source parameter can indicatewhere in the automobile the button is located (e.g., on the dashboard,on the steering wheel, on the center console); this parameter mayidentify a group of buttons (or other controls) rather than a singlebutton. In this embodiment, accessory 220 determines the mapping ofcontrols to the GUI navigation buttons of the bitmask and sets theappropriate bit (or bits) in the bitmask to indicate the controloperation.

The RotationInput command can be used to communicate information aboutthe operation of a rotational input control such as a free wheel or jogwheel. The payload can include a control-source parameter thatidentifies which rotational control was operated and additionalparameters describing the operation.

The control-source parameter can be defined in various ways. Inembodiments where accessory 220 provides information about its controlsin advance of their use (e.g., using the AccessorySettings orSetControlInfo command as described above), the control-source parametercan be the control identifier assigned by accessory 220. In otherembodiments, accessory 220 is not required to provide information aboutthe controls in advance of use. In such embodiments, the control-sourceparameter can simply identify the approximate location of the controlthat was operated. For instance, in one embodiment where accessory 220is an automobile head unit, the control-source parameter can indicatewhere in the automobile the rotational control is located (e.g., on thedashboard, on the steering wheel, on the center console) and the type ofcontrol (e.g., free wheel or jog wheel).

The type of operation can be characterized using additional parameters.For example, the direction of rotation (e.g., left/right, or up/down)can be indicated by a parameter. The particular action, e.g., whetherthe rotation has ended or is in progress, can also be indicated. For jogwheels, rotation can be considered as “in progress” for as long as theuser continues to hold the wheel away from the home position even if thecontrol is not actually moving. Alternatively, a separate action typecan be defined to indicate that the user is holding the wheel in aposition corresponding to the indicated direction of rotation.

In some embodiments, the amount of rotation of a free wheel or jog wheelduring a relevant time interval can be measured and reported. Forinstance, some knobs provide a series of detents, and the amount ofrotation can be reported as the number of detents traversed as afraction of the total number of detents available. Alternatively, theamount of rotation can be measured in degrees or the like.

As noted above, the RotationInput command can be sent at intervals whilethe user continues to operate the control. Where the amount of rotationis reported, each instance of the command can report the amount ofrotation since the last command was sent. When the rotational actionends, accessory 220 can send a final RotationInput command indicatingthat the operation has stopped.

The TouchInput command can be used to communicate information about theoperation of a touch-sensitive input control. The payload can include acontrol-source parameter identifying which control was operated andadditional parameters describing the operation.

As with other input commands described herein, the control-sourceparameter can be defined in various ways. For example, in embodimentswhere accessory 220 provides information about the controls in advanceof use (e.g., using the AccessorySettings or SetControlInfo command asdescribed above), the control-source parameter can be the controlidentifier assigned by accessory 220. In other embodiments, accessory220 is not required to provide information about the controls in advanceof use. In such embodiments, the control-source parameter can simplyidentify the approximate location of the control that was operated. Forinstance, in one embodiment where accessory 220 is an automobile headunit, the control-source parameter can indicate where in the automobilethe touch-sensitive control is located (e.g., on the dashboard, on thecenter console) as well as the type of touch-sensitive input control(e.g., single-touch or multitouch).

The type of operation can be characterized, e.g., by coordinatesindicating the location that was touched. In one embodiment, coordinatescan be specified using position indexes in the horizontal and verticaldirections (e.g., starting from top left of the touch-sensitive area),and accessory 220 can provide the maximum index along with the actualindex of the touched location. Alternatively, the touched location canbe identified using pixel coordinates, physical measurements (e.g.,inches from the screen's edge), or the like. In some embodiments using amultitouch input control, accessory 220 can interpret a user action as a“gesture,” e.g., tap, pinch, swipe, etc.; and the type of operation canbe characterized by the gesture type as well as coordinates touched,speed, number of fingers used, and the like.

As with other input commands described herein, accessory 220 can sendTouchInput commands at regular intervals as long as a touch operationcontinues. When the operation ends, accessory 220 can send a finalTouchInput command indicating that the operation has stopped.

The EnterRemoteGUIMode and ExitRemoteGUIMode commands described abovecan allow accessory 220 to switch PMD 202 into or out of remote GUImode. In some embodiments, PMD 202 may have more than two user interface(“UI”) modes. For example, PMD 202 can have a standard operating mode inwhich it presents its native GUI on its own display, a remote GUI modein which it delivers GUI images to accessory 220, and an external UImode in which an accessory provides a local GUI for controlling PMD 202and PMD 202 disables its native GUI.

Where more than two UI modes are provided, it can be useful foraccessory 220 to determine and select the desired mode. Accordingly, thecommands in table 400 include a GetUIMode command, a RetUIMode command,and a SetUIMode command. Accessory 220 can send the GetUIMode command todetermine the current UI mode, and PMD 220 can respond with theRetUIMode command, whose payload includes an identifier of the currentUI mode. To change the UI mode, accessory 220 can send the SetUIModecommand with a payload identifying the desired mode.

It will be appreciated that the commands described herein areillustrative and that variations and modifications are possible.Commands from tables 300 and 400 can be used in combination with eachother, and other commands may be added or substituted. In someembodiments, an acknowledgement (Ack) command can be sent by the PMD inresponse to any command from the accessory that does not require anotherresponsive command. The Ack command can have a payload including astatus indicator, which can be used to communicate errors.

In some embodiments, PMD 202 can maintain a list of configuration datafor known remote-GUI-enabled accessories, e.g., in storage device 206.Each configuration can be associated with a particular accessoryidentifier, such as manufacturer plus model name, an arbitrarilyassigned code, or the like. Instead of sending configuration informationcommands as described above, accessory 220 can send an identificationcommand that provides the accessory identifier. PMD 202 can then accessthe stored configuration data for the accessory. This can result infaster and more efficient setup.

In other embodiments, the first time a particular accessory such asaccessory 220 connects to PMD 202, the accessory can use theSetDisplayInfo and SetControlInfo commands described above to establishits configuration. PMD 202 can then assign the accessory a uniqueidentifier, provide the assigned identifier to the accessory, and storethe configuration information in storage device 206 in association withthe identifier. On subsequent reconnections, the accessory can send anidentification command with the unique identifier to PMD 202, and PMD202 can access the stored configuration information.

Other commands may also be included, such as commands associated withidentifying PMD 202 to accessory 220 (or vice versa), authenticatingaccessory 220 to PMD 202 (or vice versa), and providing additionalinformation about the capabilities of PMD 202 to accessory 220. Forexample, using appropriate commands, accessory 220 can determine thespecific model and/or serial number of PMD 202, whether a particular PMD202 supports a remote GUI at all, and (where there are multiple possibleversions of the remote GUI functionality), which version of remote GUIfunctionality the particular PMD 202 supports. Accessory 220 can usethis information in determining whether to enter remote GUI mode with aparticular PMD 202.

In any of the above-described commands, information can be sent as astructured data field, e.g., with certain bytes associated with certaininformation types. Alternatively, the command may include a bitmaskparameter used to identify the type of information being delivered, andthe data can be interpreted by the recipient in accordance with thebitmask.

Further, the set of commands can also include commands sent by PMD 202to request any of the available information types, as well as commandssent by PMD 202 to acknowledge receipt of the information-providingcommands from accessory 220.

In some embodiments, some information can be sent by accessory 220either in response to a request from PMD 202 or without waiting for arequest, e.g., in response to changed conditions. For instance, ifaccessory 220 is installed in a vehicle with automatic transmission,accessory 220 might send a SetStatusInfo command when it detects thatthe vehicle is shifted into or out of Park. Alternatively, accessory 220might monitor the vehicle's speed and send a SetStatusInfo commandreporting the current speed, either periodically or when it detects thatthe vehicle's speed crosses above or below a predetermined threshold. Inone embodiment, video operation can be enabled or disabled depending onwhether the vehicle's speed is above or below the threshold; thethreshold can be set to zero or a slow speed such as 5 miles per hour.

Depending on implementation, either the accessory or the PMD candetermine which functionalities of the PMD are to be exposed through theremote GUI. As described above, the accessory can send commands toselect the functionality that should be presented, or the PMD can selectfunctionality based on the current status of the accessory.

FIG. 5 is a flow diagram of process 500 that can be used to provide aremote GUI for PMD 202 on accessory device 220 according to anembodiment of the present invention. Process 500 starts (block 502) whenaccessory 220 becomes connected to PMD 202. At block 504, accessory 220is identified and authenticated. Conventional techniques for identifyingand authenticating an accessory, e.g., by exchanging commands andassociated data, can be used. Block 504 can also include determiningwhether remote GUI mode is to be entered; for example, accessory 220 cansend the EnterRemoteGUIMode command or the SetUIMode command describedabove.

At block 506, accessory 220 can provide configuration, environmentand/or status information to PMD 202. In one embodiment, any combinationof the SetDisplayInfo, SetControlInfo, SetEnvInfo, AccessorySettings,and SetStatusInfo commands described above can be used. Accessory 220can also provide an accessory-specific (or environment-specific) imageelement to PMD 202, e.g., using the SetEnvImage command described above.In some embodiments, some or all of the configuration and environmentinformation for accessory 220 can be pre-stored in PMD 202 as describedabove, and block 506 may include PMD 202 using the accessoryidentification obtained during block 504 to access the pre-storedconfiguration and environment information. Accessory 220 can alsosupplement or override any pre-stored information with new informationusing the commands described above.

At block 508, PMD 202 generates an initial remote GUI image using theinformation provided at block 506. In one embodiment, PMD 202 can applyvarious rules to adapt the native GUI image to the configuration ofaccessory 220 and/or to the status or environment of accessory 220. Suchrules can be incorporated into a control program executed by processor204 to generate remote GUI images.

For instance, the remote GUI can be adapted to the particularconfiguration of the accessory's display 232. For example, the size (inpixels or physical dimensions) of accessory display 232 of FIG. 2 (orthe portion thereof allotted to the remote GUI) might be different fromthe size of PMD display 209. PMD 202 can modify the native GUI image bychanging the size of image elements, changing the number of elementsdisplayed, or both. Further, the aspect ratio (height to width) ofaccessory display 232 may be different from that of PMD display 209, andPMD 202 may arrange the elements of the remote GUI image to optimize useof the available space. In still another example, the color depth ofaccessory display 232 might also be different from that of PMD display209, and PMD 202 can adjust the color settings for various imageelements to produce a visually pleasing appearance.

Additionally, as noted above, accessory 220 may supply an image elementto PMD 202. PMD 202 can determine a size and position for theaccessory-supplied image element with in the remote GUI image. Theelement might be small, such as a logo in a corner of the display area,or it can be larger as desired.

In embodiments where accessory 220 provides information about its inputcontrols 222 in advance of use, PMD 202 can adapt the remote GUI imageto the particular configuration of the accessory's user input controls222. For example, if accessory input control 222 provides a touch screen(e.g., as shown in FIG. 1C), GUI elements can be placed anywhere withinthe active area of the touch screen. If accessory input control 222provides an array of buttons (e.g., as shown in FIG. 1B), active GUIelements can be placed near the button that is mapped to that element.PMD 202 can determine the mapping of GUI elements to buttons or otherinput controls and arrange the elements accordingly within the displayarea.

As described above, in some embodiments, accessory 220 need not provideany configuration information for input controls 222 in advance of theiruse. In such embodiments, PMD 202 can configure the remote GUI imagewithout regard to the physical location of input controls 222. Theremote GUI image can be organized to facilitate intuitive navigationwith up/down and/or left/right buttons, e.g., by arranging selectableitems in a list or grid and providing a cursor that visuallydistinguishes one item as being the current selection (i.e., the itemthat will be activated if the user presses a select button).

In some embodiments, PMD 202 can adapt the remote GUI to the environmentof accessory 220. For example, if accessory 220 is incorporated into adashboard console for an automobile, it may be desirable to adapt theremote GUI image for ease of use by a driver. Accordingly, PMD 202 canselect different fonts or font sizes to make remote GUI elements anddisplayed information larger and easier to read. PMD 202 may alsosimplify the GUI, e.g., omitting animations or reducing the numberand/or complexity of GUI elements on a single screen, to minimize driverdistraction. In contrast, for an accessory that is mounted on anexercise machine, user distraction may be less of a concern, and PMD 202can provide complex or animated remote GUI images.

In another example, the native GUI of PMD 202 may use a light backgroundwith dark text and highlighting. This can provide good visibility indaylight, but when driving at night, the resulting brightness might bedistracting. In some embodiments PMD 202 can adapt the color schemedepending on whether accessory 220 is in day or night mode—e.g.,selecting bright text and dark backgrounds for nighttime use, or darktext and bright backgrounds for daytime use. Day or night mode can beselected based on status information provided by accessory 220, or insome embodiments, the remote GUI can incorporate a user-settable optionto select day or night mode.

As another example, PMD 202 may use environment and/or statusinformation to determine which options or functionalities should be madeavailable (exposed) in the remote GUI. For example, PMD 202 may providevideo playback capability and may be able to send video content toaccessory 220 for display. But playing video on a console visible to thedriver of a moving automobile is hazardous. To mitigate this hazard,accessory 220 can provide status information indicating, e.g., whetherthe automobile is in motion (or in gear) at a given time. PMD 202 canuse this status information to determine whether it is safe to allowvideo playback. If not, video playback options can be omitted from theremote GUI image or shown in a disabled state. Thus, for instance, PMD202 can enable video playback only when an automobile is in “Park,” onlywhen the parking brake is engaged, or only when the vehicle's speed isbelow a particular threshold (such as 5 miles per hour). Similarly, anyother functionalities supported by PMD 202 can be selectively hidden orexposed in response to environmental status information.

As yet another example, in some embodiments, accessory 220 can specifythe functionalities or options that should be made available in theremote GUI, e.g., using the SetPMDOptions command described above. Ingenerating the remote GUI image, PMD 202 can limit the elementsdisplayed to those that correspond to active functionalities or options.Inactive functionalities or options can be omitted from the remote GUIimage entirely or included with a visual indication signifying that theyare inactive (e.g., image elements corresponding to inactivefunctionalities can be grayed-out, darkened, lightened, or otherwisevisually de-emphasized).

In some embodiments, PMD 202 can incorporate status information into theremote GUI. For example, if accessory 220 is incorporated into a consolefor an exercise machine, accessory 220 can provide PMD 202 withinformation about a workout in progress as noted above. PMD 202 canincorporate this information into the remote GUI image, allowing a userto monitor the progress of his or her workout while manipulating theremote GUI or playing video content. In embodiments where accessory 220provides an image element (such as a logo), PMD 202 can incorporate theimage element into the remote GUI. For example, PMD 202 may beprogrammed to reserve a portion of a remote GUI image for anaccessory-supplied image element and may insert the accessory-suppliedelement into the reserved portion in any or all remote GUI images. PMD202 can resize the accessory-supplied image element as needed to make itfit within the reserved area. (If the accessory does not supply an imageelement, the reserved portion can be left blank or filled with a defaultimage element.)

Referring again to FIG. 5, at block 510, PMD 202 can send the remote GUIimage to accessory 220. For example, PMD 202 can send the image using ananalog or digital video output interface provided by accessory I/Ointerface 214 and PMD I/O interface 226 of FIG. 2. (The same videointerface can also be used for delivering video content to accessory220.) Alternatively, PMD 202 can use the RemoteGUIImageData commanddescribed above. At block 512, accessory 220 can display the remote GUIimage. In some embodiments, accessory 220 displays the image as receivedand does not modify it; thus, PMD 202 can control the look and feel ofthe remote GUI. Blocks 510 and 512 can be repeated until user input isdetected; for example, PMD 202 can repeatedly send the same remote GUIimage at the refresh rate of the display device of accessory 220.

At block 514, accessory 220 detects user operation of one of inputcontrols 222. At block 516, accessory 220 can forward the user input toPMD 202, e.g., using the ProcessUserInput command or the type-specificinput commands (e.g., ButtonInput, RotationInput, TouchInput) describedabove. As noted above, the command can indicate which control wasoperated and, if applicable, the nature of the operation. Accessory 220can forward the user input without processing it to identify aresponsive action to be taken; instead, accessory 220 can simply forwardan identification of the detected user action.

At block 518, PMD 202 can process the user input to determine anoperation (or action) to be performed in response. At block 520, if theoperation does not correspond to exiting the remote GUI mode, process500 proceeds to block 522, where PMD 202 performs the operationrequested by the user. Any operation made accessible through the remoteGUI image can be performed, such as starting playback, pausing playback,adjusting settings, browsing or searching a database, and so on.Performing some operations may involve sending commands from PMD 202 toaccessory 220 (e.g., to change settings such as volume, displaybrightness, etc.). Performing other operations (e.g., playback) mayinvolve sending media content from PMD 202 to accessory 220. Still otheroperations (e.g., database queries) can be performed by PMD 202 withoutsending commands or content to accessory 220. At block 524, PMD 202 cangenerate an updated remote GUI image based on the selected operation.Process 500 can then return to block 512 to display the updated remoteGUI image. It should be noted that some operations, such as playback,can be ongoing, and a playing track (or sequence of tracks) can continueto play while process 500 iterates. (Playback can be stopped, e.g., bythe user selecting a pause or stop operation, which selection can beprocessed using process 500.)

Process 500 can continue until such time as a user selects an operationthat entails exiting remote GUI mode at block 520. At that point,process 500 ends (block 526). In some embodiments, PMD 202 can signalaccessory 220 that the remote GUI mode is ending, e.g., using theExitRemoteGUIMode command described above. Thereafter, accessory 220 canreturn to its local GUI operating mode.

In some embodiments, PMD 202 can remain connected to accessory 220 afterexiting remote GUI mode. Thus, PMD 202 can continue to send mediacontent to accessory 220, so that a playing track can continue to playeven after remote GUI mode is exited. Thus, for example, if accessory220 is incorporated into a console of an exercise machine, the user mayexit the remote GUI to change workout settings while a song or videocontinues to play. As another example, if accessory 220 is incorporatedinto an automobile dashboard console that also provides navigationfunctionality, the user can exit remote GUI mode to perform navigationoperations while music continues to play.

Further, in some embodiments, accessory 220 can support a user inputthat, when selected, instructs accessory 220 to re-enter remote GUImode. Because PMD 202 remains connected, accessory 220 can send anEnterRemoteGUIMode command in response to detecting this user input.Thus, the user can enter and exit the remote GUI mode as desired,controlling PMD 202 through the remote GUI mode and controlling otherfeatures of accessory 220 through a local user interface.

In other embodiments, accessory 220 can provide source-selection inputcontrols that allow a user to select among multiple sources of audioand/or video signals. For example, accessory 220 can be implemented inan in-vehicle head unit that also supports AM, FM, and/or satelliteradio receivers; a GPS-based navigation system; environmental controls(e.g., adjusting a cabin thermostat); and so on, in addition to theremote GUI functionality provided by PMD 202. The head unit may providebuttons (or other user input controls) associated with selecting amongthese functionalities. The head unit may, but need not, map such buttonsto the remote GUI. For instance, if the user operates a button torequest an alternate audio source (e.g., FM radio) during remote GUIoperation, the head unit can interpret the request and switch the audiosource. In some embodiments, the head unit can notify PMD 202 of thechange, e.g., using a SetStatusInfo command as described above. When theaudio and/or video source is switched away from PMD 202, PMD 202 cancontinue to generate the remote GUI image or not, depending onimplementation. In one embodiment, PMD 202 continues to generate theremote GUI image regardless of audio source selection but discontinuesgenerating the remote GUI image if the video source selection isswitched away from PMD 202.

It will be appreciated that process 500 is illustrative and thatvariations and modifications are possible. Steps or blocks described assequential may be executed in parallel, order of steps may be varied,and steps may be modified or combined. Commands and communicationprotocols other than those described herein can be implemented to allowa portable media device to provide a remote GUI image to an accessoryand to process user input responsive to the remote GUI image.

FIG. 6 is a flow diagram of a process 600 for operating an accessory(e.g., accessory 220 of FIG. 2) in conjunction with a PMD (e.g., PMD 202of FIG. 2) that provides a remote GUI according to an embodiment of thepresent invention. Process 600 starts (block 602) when accessory 220becomes connected to PMD 202. At block 604, accessory 220 can sendidentification and authentication information to PMD 202. In someembodiments, the identification information can include an indicationthat accessory 220 is capable of using the remote GUI functionality ofPMD 202 and can also include configuration information for theaccessory's display and/or input controls. For example, theAccessorySettings command and/or SetDisplayInfo and SetControlInfocommands described above can be used. Authentication, where provided,may include exchanging digital certificates and/or cryptographicmessages with PMD 202. In some embodiments, authentication can beunidirectional (e.g., accessory 202 authenticates itself to PMD 202 orvice versa) or bidirectional (e.g., each device authenticates itself tothe other). During identification and authentication, accessory 220 andPMD 202 can also exchange messages regarding which version of a remoteGUI protocol and/or supporting software program each implements.

If identification or authentication fails at block 604, process 600 canexit (not explicitly shown). For example, PMD 202 can decline furthercommunication with accessory 220 if accessory 220 fails to authenticateitself. As another example, if the version of a remote GUI generationprogram used by PMD 202 is not compatible with or not approved byaccessory 220, accessory 220 can determine not to enter remote GUI mode.

If, however, identification and authentication at block 604 succeed,process 600 proceeds to block 606. At block 606, accessory 220 canobtain options related to the remote GUI from PMD 220, e.g., using theGetPMDOptions and RetPMDOptions commands. For example, accessory 220 cansend the GetPMDOptions command to determine which functionalities PMD202 is capable of exposing via the remote GUI. At block 608, accessory220 can select one or more functionalities that should be exposed, e.g.,using the SetPMDOptions command.

At block 610, accessory 220 can instruct PMD 202 to enter remote GUImode, e.g., using the EnterRemoteGUIMode or SetUIMode command describedabove. In some embodiments, accessory 220 automatically instructs PMD202 to enter remote GUI mode after establishing that a particular PMD iscompatible with the accessory's level of support for remote GUI mode. Inother embodiments, accessory 220 can wait for user input or anotherevent indicating that remote GUI mode should be entered beforeinstructing PMD 202 to enter remote GUI mode.

At block 612, accessory 220 receives and displays a video signal fromPMD 202; in general, this video signal can provide a remote GUI screenimage and/or other image(s). For example, if PMD 202 is being operatedto play back video content, the signal can include the video content. IfPMD 202 is being operated to display stored still images (e.g., photos),the signal can include a still image.

In some embodiments, the video signal is an analog signal intended tocover the entire screen, and accessory 220 can simply generate displaypixels in response to the video signal. In other embodiments, the remoteGUI image covers only a portion of the screen, and accessory 220 canblend the video signal from PMD 202 with a signal from another sourcethat provides the image for the rest of the screen. Digital videosignals in various formats can also be used.

At block 614, accessory 220 determines whether user input is beingreceived. If so, accessory 220 processes the user input. Morespecifically, at block 616, accessory 220 determines whether the inputis “local” input. For example, as described above, accessory 220 mayreserve some of the input controls as not being mapped to remote GUIfunctions, such as controls allowing the user to switch audio or videosources, or the like. Such reserved controls provide “local” input thataccessory 220 can process rather than forwarding to PMD 202. If suchlocal input is received, then the local input is processed by accessory220 at block 618.

At block 620, accessory 220 determines whether the remote GUI modeshould be exited in response to the local input. If so, at block 622,accessory 220 instructs PMD 202 to exit remote GUI mode, e.g., using theSetUIMode or ExitRemoteGUIMode commands described above, and process 600ends (block 624). Otherwise, process 600 returns to block 612 andaccessory 220 continues to receive and display a video signal from PMD202.

If, at block 616, user input other than local input is detected,accessory 220 reports the user input to PMD 202 at block 626. Dependingon implementation, the ProcessUserInput command or a type-specific inputcommand (e.g., ButtonInput, RotationInput, TouchInput commands)described above can be used to report user input to PMD 202. Thesecommands can indicate that a particular input event occurred whileallowing PMD 202 complete control over how the event is interpreted andwhether it has any effect on the displayed remote GUI image.

As described above, in some embodiments, accessory 220 continues to sendcommands reporting user input to PMD 202 for as long as the usercontinues to operate a control and sends a final command to indicatethat the operation has ended. The reported user input may or may notaffect the appearance of the remote GUI; any changes can be made by PMD202 without any intervention by accessory 220 (other than reporting theuser input). In addition the reported user input may result in PMD 202invoking its own functionality, e.g., playing a track, initiating aphone call, or the like; again, accessory 220 does not need to interpretthe user input or determine what functionality to invoke.

Regardless of whether user input is detected at block 614, process 600can also check at block 628 to determine if any state changes haveoccurred in the operating environment of accessory 220. For example, ina vehicle, if the headlights are turned on or off, the changed state ofthe headlights can be a detectable change in the operating environment.In a console of an exercise machine, the beginning or ending of aworkout can be a detectable change. As yet another example, as describedabove, accessory 220 can detect when the user changes the selected inputsource for audio and/or video. At block 630, accessory 220 can report adetected state change to PMD 202, e.g., using the SetStatusInfo commanddescribed above. As with user input, status changes may result in PMD202 modifying the remote GUI image and/or invoking its ownfunctionality. Accessory 220 does not need to determine or specify what(if any) effect a particular state change should have; it can simplyreport various state changes to PMD 202.

In some embodiments, accessory 220 can also respond to a detected statechange by determining that the selection of PMD functionality to beexposed via the remote GUI should be changed. For example, if accessory220 is an in-vehicle head unit, accessory 220 may be able to determinewhen the vehicle transitions into or out of a “parked” state (e.g., bydetecting when an automatic transmission is shifted into or out of Parkor by detecting when a parking brake is engaged or disengaged). In someembodiments, accessory 220 can allow certain functionality that isunavailable when the vehicle is not parked (e.g., video playback orthird-party application programs that may cause unacceptable driverdistraction) to become available when the vehicle is parked.Accordingly, notifying PMD 202 of a state change at block 630 caninclude sending a new SetPMDOptions command to change the selection offunctionalities to be exposed via the remote GUI in addition to orinstead of providing another state-change notification.

Process 600 can continue indefinitely while accessory 220 continues toreceive and display video signals from PMD 202 and to report user inputand state changes to PMD 202. Process 600 can end, e.g., when accessory220 determines at block 620 that remote GUI mode should be exited. Onceprocess 600 ends, it can be restarted, e.g., when accessory 220reconnects to PMD 202. In some embodiments, accessory 220 can provide auser input control that can be operated to re-enter the remote GUI modewithout requiring disconnection and reconnection; such an accessory canre-enter process 600, e.g., at step 610, as long as the accessoryremains connected to the PMD.

It will be appreciated that process 600 is illustrative and thatvariations and modifications are possible. Operations described assequential may be executed in parallel, order of operations may bevaried, and operations may be modified, combined, added or omitted. Forinstance, those skilled in the art will appreciate that while a cycle ofoperations to check for user input and state changes may be suggested bythe flow diagram, other implementations (e.g., based on interrupts) arepossible. In some embodiments, PMD 202 can determine under certainconditions that remote GUI mode should be exited and exit remote GUImode without receiving an exit command from accessory 220. Depending onimplementation, PMD 202 might or might not notify accessory 220 when PMD202 decides to exit remote GUI mode.

FIG. 7 is a flow diagram of a process 700 for remote GUI operation thatcan be executed in a PMD (e.g., PMD 202 of FIG. 2) according to anembodiment of the present invention. Process 700 starts (block 702) whenan accessory (e.g., accessory 220 of FIG. 2) becomes connected to PMD202. At block 704, PMD 202 can receive identification and authenticationinformation from accessory 220. In some embodiments, the identificationinformation can include an indication that accessory 220 is capable ofusing the remote GUI functionality of PMD 202 and can also includeconfiguration information about the accessory's display and/or inputcontrols. For example, the AccessorySettings command and/orSetDisplayInfo and SetControlInfo commands described above can be used.Authentication, where provided, may include exchanging digitalcertificates and/or cryptographic messages between PMD 202 and accessory220. In some embodiments, authentication can be unidirectional (e.g.,accessory 202 authenticates itself to PMD 202 or vice versa) orbidirectional (e.g., each device authenticates itself to the other).During identification and authentication, PMD 202 and accessory 220 canalso exchange messages regarding which version of a remote GUI protocoland/or supporting software program each implements.

If identification or authentication fails at block 704, process 700 canexit (not explicitly shown). For example, PMD 202 can decline furthercommunication with accessory 220 if accessory 220 fails to authenticateitself.

After successful identification and authentication, process 700 canproceed to block 706. At block 706, PMD 202 can provide informationrelated to remote GUI options to accessory 220. In some embodiments, theinformation can be provided in response to a request from accessory 220;for instance, PMD 202 can receive the GetPMDOptions command describedabove and provide the options using the RetPMDOptions command describedabove. At block 708, PMD 202 can receive instructions from the accessoryregarding functionalities to be exposed in the remote GUI; for example,PMD 202 can receive a SetPMDOptions command from accessory 220.

At block 710, PMD 202 can enter remote GUI mode. In some embodiments,PMD 202 enters remote GUI mode only on instructions from accessory 220.For example, PMD 202 can receive the EnterRemoteGUIMode or SetUIModecommand described above. In some embodiments, if the version of a remoteGUI generation program used by PMD 202 is not compatible with or notapproved by accessory 220, accessory 220 can determine not to enterremote GUI mode, in which case process 700 would not proceed past block710.

At block 712, PMD 202 generates a remote GUI image and transmits acorresponding video signal to the accessory. In some embodiments, thevideo signal can be transmitted using analog or digital video outputpins of PMD 202; in other embodiments, the RemoteGUIImageData commanddescribed above can be used.

Generation of the remote GUI image can be similar to that describedabove with reference to FIG. 5. In some embodiments, the remote GUIimage for any particular operating condition is based on the PMD'snative GUI image. The native GUI image can be modified to take intoaccount characteristics of the accessory's display, such as size (inpixels or physical dimensions), aspect ratio, color depth, and the like.An image element supplied by the accessory can also be incorporated intothe remote GUI image. In some embodiments, the image can be modified toreflect the location of accessory input controls; however, as describedabove, this is not required. The remote GUI image can also be adapted tothe operating environment of a particular accessory (e.g., making GUIelements larger and/or easier to read if the accessory is in anautomobile or other vehicle) and/or to the operating status of theaccessory (e.g., day or night mode). The selection of whichfunctionality to expose can be based on environment and/or statusinformation, or the selection can be based on explicit instructions fromthe accessory (e.g., a SetPMDOptions command).

At block 714, PMD 202 determines whether a command has been receivedfrom accessory 220. If not, PMD 202 can continue to generate remote GUIimages (or simply resend the same remote GUI image) at block 712 until acommand is received. If a command is received at block 714, PMD 202processes the command.

More specifically, at block 716, PMD 202 determines whether the receivedcommand indicates that remote GUI mode should be exited (e.g., theExitRemoteGUIMode or SetUIMode commands described above). If so, PMD 202stops generating remote GUI images at block 718, and process 700 ends(block 720).

If, at block 716, the received command does not indicate that remote GUImode should be exited, then at block 722, PMD 202 determines whether thecommand indicates user input received by accessory 220. For example, PMD202 can receive the ProcessUserInput command or any of the type-specificinput commands (e.g., ButtonInput, RotationInput, TouchInput commands)described above, or other commands indicative of user input. If userinput is received, then at block 724 PMD 202 processes the user input.

Processing the user input at block 724 can include a number of differentoperations depending on the specific input received and the currentstate of the remote GUI. For example, if the remote GUI is currentlydisplaying a list of items, one of which can be highlighted, certainuser input (e.g., operation of up or down buttons, turning of a wheel orknob, a swipe gesture on a touch screen) can be interpreted as scrollingthe list or moving the highlighting to a different item. PMD 202 canupdate the remote GUI image to show a different highlighted item or adifferent portion of the list (or both) based on the user input.

Similarly, if the remote GUI is currently displaying a list of items,certain user input (e.g., operation of a select button, releasing of awheel or knob, a tap gesture on a touch screen) can be interpreted asselecting an item (e.g., the currently highlighted item). PMD 202 candetermine which item is being selected and invoke a functionalityassociated with the selected item. For example, if the selected item isa media track, PMD 202 can initiate playback of the track; if theselected item is an icon associated with a PMD functionality (e.g.,telephone), PMD 202 can invoke that functionality; if the selected itemis an icon associated with a particular application, PMD 202 can launchthe application; and so on. Thus, processing the user input at block 724can include updating the remote GUI image to reflect the user inputand/or invoking functionality of PMD 202 based on the user input. Afterprocessing the user input, PMD 202 can continue to transmit the remoteGUI image (with any applicable updates) at block 712.

If, at block 722, the received command is not a user input command, thenat block 726 PMD 202 can determine whether the received command isindicative of a state change in the accessory, e.g., the SetStatusInfocommand described above. If so, then PMD 202 can process the statechange at block 720. Depending on the particular state change indicated,processing of the state change might or might not include updating theremote GUI image. For example, if the state change indicates atransition from day to night mode, the remote GUI image can be updatedto reflect a “night-mode” color scheme. If the state change indicatesthat the accessory is switching away from using PMD 202 as a video inputsource, PMD 202 can suspend updating and/or transmitting remote GUIimages until a subsequent state change command indicates that theaccessory is switching back to using PMD 202 as a video input source.Other state changes can affect the selection of functions to be exposedvia the remote GUI (e.g., the PMD can receive a new SetPMDOptionscommand or can determine based on the state-change information that theselection should change), and the remote GUI image can be updated toreflect the currently available functions. Still other state changes canaffect ongoing operations within the PMD. For example, in an embodimentwhere PMD 202 is used with an in-vehicle head unit as accessory 220, PMD202 can automatically pause or exit video playback if accessory 220signals that the vehicle has been taken out of a parked state. Afterprocessing the state change, PMD 202 can display the remote GUI image(with any applicable updates) at block 712.

If, at block 726, it is determined that the received command is not astate change command, then at block 730, the command is processed. Insome embodiments, some commands are ignored during remote GUI modeoperations; for example, commands requesting information about storedcontent on PMD 202 can be ignored. (The user can access such informationvia the remote GUI without PMD 202 needing to supply any information tothe accessory.) Other commands, however, can be accepted and processedat block 730. For example, some accessories have buttons associated withplayback control (e.g., play/pause, fast-forward, and rewind buttons),and commands invoking PMD functions associated with playback control canbe processed at block 730, thereby allowing a user to use theaccessory's buttons in an intuitive way.

Process 700 can continue indefinitely, with PMD 202 continuing togenerate and transmit remote GUI images that it updates based on commandinput from accessory 220 and to invoke its internal functions based onuser input relayed by accessory 220. Process 700 can end, e.g., when PMD202 receives a command indicating that remote GUI mode should be exitedor when accessory 220 becomes disconnected from PMD 202. Once process700 ends, it can be restarted, e.g., when PMD 202 reconnects toaccessory 220. In some embodiments, after exiting remote GUI mode, PMD202 can accept a subsequent command from accessory 220 to re-enterremote GUI mode without requiring disconnection and reconnection; insuch embodiments, PMD 202 can simply re-enter remote GUI mode at block710.

It will be appreciated that process 700 is illustrative and thatvariations and modifications are possible. Operations described assequential may be executed in parallel, order of operations may bevaried, and operations may be modified, combined, added or omitted. Forinstance, those skilled in the art will appreciate that while a cycle ofoperations to check for user input and state changes may be suggested bythe flow diagram, other implementations (e.g., based on interrupts) arepossible. In some embodiments, PMD 202 can determine under certainconditions that remote GUI mode should be exited and exit remote GUImode without receiving an exit command from accessory 220. Depending onimplementation, PMD 202 might or might not notify accessory 220 when PMD202 exits remote GUI mode.

As noted above, a PMD can expose any of its functionalities via theremote GUI. FIG. 8 illustrates a native GUI on a PMD 800 that has anumber of different functionalities. Native GUI image 802 includes anumber of icons displayed on a touchscreen 804 of PMD 800. The user cantouch an icon to select the associated functionality. For example, icon806 is associated with media player functionalities (including audio andvideo playback); icon 808 is associated with a web browser; icon 810with telephony; icon 812 with maps and navigation aids; and so on.

FIG. 9A illustrates a remote GUI image associated with the native GUI ofPMD 800 according to an embodiment of the present invention. Remote GUIimage 900 is displayed on a display screen 902 of an accessory 904 thathas associated control buttons 906 a-e (e.g., up, down, left, right,select). Remote GUI image 900 includes icons that look generally likethe icons of native GUI image 802, and the user can activatefunctionalities of PMD 800 by selecting the corresponding icon usingcontrol buttons 906 a-e. In the embodiment shown, one of the icons ishighlighted with cursor 908 to indicate that it will be activated if theuser presses select (SEL) button 906 e. Thus, icon 910 (media player) iscurrently highlighted. A user can change the selection by operatingarrow buttons 906 a-d to move cursor 908 up, left, right, or down andselect an icon using the select (SEL) button 906 e.

In the embodiment shown, not all functionalities of PMD 800 are exposedvia the remote GUI, and icons corresponding to functionalities notexposed are not drawn in remote GUI image 902. Thus, for example, webbrowsing (icon 808) is not available. Other embodiments may provide theicons for unexposed functionalities in grayed-out or otherwisede-emphasized form so that the user can readily determine whichfunctionalities are available for selection.

FIG. 9B illustrates a second remote GUI image 950 according to anembodiment of the present invention. Image 950 provides a selection menufor media player functionalities. In some embodiments, PMD 800 cangenerate image 950 when a user selects media player icon 910 in image900 of FIG. 9A. Image 950 provides media playback options. Similarly toimage 900 described above, image 950 provides a cursor 952 that the usercan move by pressing buttons 906 a-d; the item currently highlighted bythe cursor can be selected by pressing select button 906 e. In theexample shown, video playback is not available (e.g., because accessory900 has instructed PMD 800 not to expose this functionality); this isindicated by graying out “Videos” menu item 954. Alternatively, menuitem 954 (or any other item corresponding to unexposed functionality)can simply be omitted from image 950.

It will be appreciated that the remote GUI images and controls describedherein are illustrative and that variations and modifications arepossible. For example, in some embodiments, if the accessory providesconfiguration information indicating that it has a touchscreen display,the cursor can be omitted. In some illustrated examples, the remote GUIimages closely resemble the native GUI images, but this is not required.For example, a grid of icons in a native GUI image can be presented in aremote GUI as a list, or a list of items in the native GUI image can bepresented in a remote GUI image as a grid, and so on. In someembodiments, the PMD generates remote GUI images that are similar oridentical across different accessories, thus providing a consistent userexperience. The particular functionalities shown are intended asillustrative and not limiting.

In some embodiments, the portable media device can have one or morethird-party applications resident on it in addition to the nativeapplications. A third-party application or “application” can be softwarecode that is not native to the portable media player and which isdesigned to perform some specific task or tasks. In other words, athird-party application is an application that can be installed by theuser of the portable media device and is not part of the standardconfiguration of the portable media device. For example, Pandora® is athird-party application that can be executed by various portable mediadevices for playing media items, e.g., music streamed over the Internet.Each such third-party application can have an associated graphical userinterface (GUI) that can be used to interact with the application. Thethird-party application can generate various items of information basedon its functionality. For example, a media player application such asPandora® can generate information about the media item that is beingplayed or has been queued up to be played. In some embodiments, suchinformation about the media item can include but is no limited to nameof the media item, artist name, album name, genre, total time for themedia item, time elapsed, time remaining, album artwork, etc.Collectively this information can be referred to as metadata associatedwith the media item.

After the metadata is generated by the third-party application, thismetadata can be provided to an operating system of the portable mediadevice, e.g., iOS for the iPod®, iPhone®, and iPad® line of products byApple Inc. The operating system can use this metadata received from thethird-party application to generate a remote GUI image to be sent to anaccessory that supports the remote GUI feature. The remote GUI image caninclude displayable information and can also have non-displayableinformation associated with it. For example, displayable information caninclude information that can be displayed on the display of theaccessory and/or the portable media player, e.g., name of the song oralbum artwork. Non-displayable information associated with the remoteGUI image can include resolution of the remote GUI image, size of theremote GUI image, etc.

In some embodiments, the portable media device can define a set of oneor more remote GUI image types based on the metadata provided by thethird-party application. A remote GUI image conforming to one of theremote GUI image types from the set of remote GUI image types can begenerated and communicated to an accessory based on the capabilities andpreferences of the accessory. In some embodiments, the displayedinformation in the remote GUI image can only include the name of thethird-party application associated with it. In other embodiments, thedisplayed information in the remote GUI image can include some or all ofthe metadata associated with a media item being played by thethird-party application. Thus, the portable media device can generateseveral types of remote GUI images for each third-party application thatcan include varying amount of displayed information. It is to be notedthat displayed information included in a remote GUI image can depend onthe data provided by the third-party application and other generic datathat can be generated by the portable media device. The followingdescription uses a media player third-party application referred to as“WebRadio” that can stream media over the Internet to explain thevarious embodiments. However, one skilled in the art will realize thatthe embodiments are equally applicable to any other third-partyapplication and that the embodiments described below should not beconstrued to unduly restrict them to only the disclosed third-partyapplications. A remote GUI image can be a static image or can includevideo information.

FIGS. 10A-10C illustrate three remote GUI images that can be generatedby the portable media device for the WebRadio media player applicationand displayed on an accessory 1000 according to an embodiment of thepresent invention.

Accessory 1000 illustrated in FIG. 10A displays a remote GUI image 1002of a first type on its display 1004. In this instance, the displayedinformation in remote GUI image 1002 only includes the name of thethird-party media player application, e.g., WebRadio, that is currentlybeing executed on the portable media device. In some embodiments, remoteGUI image 1002 can be displayed when the third-party application isplaying a media item. Remote GUI image 1002 can be suitable foraccessories that have minimal display capabilities, e.g., accessorieswith only a single line of display capability. However, remote GUI image1002 can also be displayed on accessories with advanced displaycapabilities if the accessory chooses.

FIG. 10B illustrates a remote GUI image 1006 of a second type. RemoteGUI image 1006 displayed by accessory 1020 includes more informationabout the media item being played by the third-party application thanremote GUI image 1002. In this instance, the information in remote GUIimage 1006 includes name of the media item 1050, name of the artist1052, and genre 1054 associated with the media item. As can be seen,remote GUI image 1006 includes more information than remote GUI image1006.

FIG. 10C illustrates a remote GUI image 1006 of a third type. Remote GUIimage 1008 displayed on accessory 1030 includes even more informationthan remote GUI image 1006. For example, remote GUI image 1008 includesthe album artwork, time elapsed, and time remaining information for themedia item being currently played. Thus, it can be seen that dependingon the accessory, various types of remote GUI images can be displayed bythe accessory.

In some embodiments, the accessory can request information aboutavailable remote GUI image types for a third-party application from theportable media device, e.g., using the GetPMDOptions command. In someembodiments, the payload of the GetPMDOptions command can include anindication that a list of remote GUI image types is requested for athird-party application and an identifier, e.g., a name or some otherunique identifier, associated with the third-party application for whichthe list of remote GUI image types is being requested. In response tothat request, the portable media device can send a list of all availableremote GUI image types for the third-party application, e.g., using theRetPMDOptions command. The payload for the RetPMDOptions command in thisinstance can include a list of available remote GUI image types for thethird-party application along with a unique identifier for each remoteGUI image type and optionally, a description of each of the remote GUIimage types. In some embodiments, the portable media device may have thelist of remote GUI image types stored on its local storage medium. Inother embodiments, the portable media device may obtain the list ofremote GUI image types from an external source, e.g., iTunes® App Storeby Apple Inc. or from the third-party application developer.

Once the accessory receives the list of the available remote GUI imagetypes, it can select one of those remote GUI image types and inform theportable media device accordingly, e.g., using the SetPMDOptions commandwith the unique identifier for the third-party application as thepayload. In response to the selection, the portable media device canthen generate and send a remote GUI image that conforms to the selectedremote GUI image type to the accessory, e.g., using theRemoteGUIImageData command or by sending a video signal. In someembodiments, the accessory manufacturer can specify which remote GUIimage type or types are supported by the accessory. Based on thispredefined specification, the accessory can select the appropriateremote GUI image type to be requested from the portable media device.

FIG. 11 is flow diagram of a process 1100 for displaying a remote GUIimage on an accessory according to an embodiment of the presentinvention. Process 1100 can be performed, e.g., by accessory 220 of FIG.2.

Process 1100 starts at block 1102. At block 1104, the accessory canidentify and authenticate itself to the portable media device. At block1106, the accessory can send a request to the portable media device,e.g., using the GetPMDOptions command, requesting the portable mediadevice to send a list of remote GUI image types available for aparticular third-party application. In response to the request, theportable media device can send a list of remote GUI image typesavailable for that particular third-party application to the accessoryat block 1108. In some embodiments, the portable media device may obtainthe list from an external system as described above.

Thereafter, the accessory can select a remote GUI image type from thelist and communicate that selection to the portable media device atblock 1110. Subsequently, the accessory can receive a remote GUI imagethat conforms to the selected remote GUI image type from the portablemedia device at block 1112. The accessory can then display the receivedremote GUI image on a display at block 1114. At block 1116, theaccessory can check whether it wants to exit the remote GUI mode, e.g.,based on instructions from a user operating the accessory ordisconnection of the accessory from the portable media device. If it isdetermined that the accessory wants to exit the remote GUI mode, process1100 can end at block 1118. If it is determined at block 1116 that theaccessory is still in the remote GUI mode, process 1100 can return toblock 1112 where the accessory receives an updated remote GUI image orkeeps receiving a continuous video signal from the portable mediadevice.

It will be appreciated that process 1100 is illustrative and thatvariations and modifications are possible. Operations described assequential may be executed in parallel, order of operations may bevaried, and operations may be modified, combined, added or omitted. Forinstance, in one embodiment, the portable media device may provide alist of third-party applications resident on the portable media deviceto the accessory prior to receiving a request from the accessory for thelist of remote GUI image types. Based on that information, the accessorycan request a list of available remote GUI image types for one or morethird-party applications and store that information in a local storageof the accessory for future use. In some embodiments, the accessory canprovide a list of third-party applications that are compatible with theaccessory or those that are permitted to be used with the accessory, tothe portable media device, e.g., as part of the identification andauthentication process. The portable media device can then use thisinformation to determine whether one or more of the compatible/permittedapplications are resident on the portable media device and if they are,the portable media device can pre-cache the remote GUI type informationfor those applications in anticipation of the request from theaccessory.

It is to be noted that actions in blocks 1106, 1108, and 1110 can occurwhether or not the portable media device is running a third-partyapplication. For example, these actions can also be performed for anynative or first-party application that is resident on the portable mediadevice.

FIG. 12 is a flow diagram of a process 1200 according to an embodimentof the present invention. Process 1200 can be performed, e.g., byportable media device 202 of FIG. 2.

Process 1200 starts at block 1202. The portable media device canidentify and authenticate the accessory at block 1204. At block 1206,the portable media device can receive a request from an accessory toprovide a list of available remote GUI image types for a particularthird-party application. As described above, the request can includeinformation about the third-party application for which the list isbeing requested. Based on the request, the portable media device candetermine a list of remote GUI image types and send the list to theaccessory at block 1208. Thereafter, the portable media device canreceive an indication from the accessory informing the portable mediadevice about which remote GUI image type was selected by the accessory,at block 1210. Once the portable media device receives the selection,the portable media device can then execute the third party applicationat block 1212. Thereafter, the portable media device can generate andsend a remote GUI image that conforms to the selected remote GUI imagetype to the accessory at block 1214. At block 1216, the portable mediadevice can check whether it has received instructions to exit the remoteGUI mode. If yes, process 1200 can then end at block 1218. If theportable media device does not receive instructions to end the remoteGUI mode at block 1216, process 1200 can return to block 1214 and theportable media device can keep sending updated remote GUI images or acontinuous video signal to the accessory.

It will be appreciated that process 1200 is illustrative and thatvariations and modifications are possible. Operations described assequential may be executed in parallel, order of operations may bevaried, and operations may be modified, combined, added or omitted. Forinstance, the accessory can communicate its display capabilities to theportable media device during the authentication and identificationprocess. Based on the display capabilities, the portable media devicecan include only those remote GUI image types in the list which arecompatible with the accessory. Thus, the portable media device canfilter out any incompatible remote GUI image types and include only theinformation for compatible remote GUI image types in the list. Forexample, consider that the accessory can only display a single line oftext. In this instance, the portable media device can excludeinformation about any remote GUI image type that includes multiple linesof display and only include information about the remote GUI imagetype(s) that include a single line of display in the list that is sentto the accessory at block 1208. It is to be noted that actions in blocks1206, 1208, and 1210 can occur whether or not the portable media deviceis running a third-party application. For example, these actions canalso be performed for any native or first-party application that isresident on the portable media device.

As described above, the portable media player can generate and send aremote GUI image associated with a third-party application based on theremote GUI image type selection provided by the accessory. In someembodiments, the accessory can request a remote GUI image that includesspecific information. In this instance, the accessory can specify theinformation items to be included in the remote GUI image. For example,based on the accessory capabilities, the accessory may be able todisplay more or less information about a media item being played by thethird-party application. Some accessories may be capable of displayingall relevant information about a media item while other accessories canhave more limited display capabilities. Thus, based on its displaycapabilities, the accessory can request the portable media device toinclude certain information items in the remote GUI image. In someembodiments, the accessory may request less information than itscapabilities depending on other factors such as safety, aesthetics, etc.In addition, the accessory can also specify other attributes, e.g.,non-displayed information, associated with the remote GUI image such asimage size, image resolution, location of the displayed informationwithin the image, etc. In some embodiments, the accessory may use amodified SetPMDOptions command to specify the information about theremote GUI image. Upon receipt of this information, the portable mediadevice can generate the remote GUI image based on the specification andsend that remote GUI image to the accessory.

In this embodiment, each remote GUI image can be made to “custom fit”the display of the accessory. Also, since each accessory can havedifferent display capabilities, the portable media device need not storeor define multiple remote GUI image types for each third-partyapplication. Instead, the portable media device can generate a customremote GUI image based on the specifications provided by each accessory.As described above, the third-party application provides various dataitems to the operating system of the portable media device. The portablemedia device can use the provided data items to construct the remote GUIimage. In this manner the remote GUI image can be dynamically generatedfor each third-party application and for each accessory.

FIG. 13 is a flow diagram of a process 1300 for displaying a remote GUIimage according to an embodiment of the present invention. Process 1300can be performed, e.g., by accessory 220 of FIG. 2.

Process 1300 starts at block 1302. At block 1304, the accessory canidentify and authenticate with the portable media device. Thereafter,the accessory can send a request to the portable media device at block1306. The request can include an indication of the information items tobe included in the remote GUI image along with other information aboutthe display capabilities of the accessory. In some embodiments, theinformation about the display capabilities of the accessory can beseparately provided to the portable media device independent of therequest in block 1306, e.g., during the authentication process of block1304. At block 1308, the accessory can receive a remote GUI image basedon the information specified in the request. The accessory can displaythe remote GUI image at block 1310. At block 1312, the accessory cancheck whether to exit the remote GUI mode. If yes, process 1300 can endat block 1314. If the accessory does not exit the remote GUI mode,process 1300 can return to block 1308 where the accessory receives anupdated GUI image or keeps receiving the video signal.

It will be appreciated that process 1300 is illustrative and thatvariations and modifications are possible. Operations described assequential may be executed in parallel, order of operations may bevaried, and operations may be modified, combined, added or omitted. Forinstance, In some embodiments, the accessory can start receiving andstop receiving remote GUI images from the portable media device based onexplicit instructions to the portable media device, e.g., using commandsEnterRemoteGUIMode and ExitRemoteGUIMode, respectively. It is to benoted that actions in blocks 1306, 1308, and 1310 can occur whether ornot the portable media device is running a third-party application. Forexample, these actions can also be performed for any native orfirst-party application that is resident on the portable media device.

FIG. 14 is a flow diagram of a process 1400 for generating a customremote GUI image according to an embodiment of the present invention.Process 1400 can be performed, e.g., by portable media device 202 ofFIG. 2.

Process 1400 begins at block 1402. At block 1404, the portable mediadevice can identify and authenticate an accessory. Thereafter, at block1406, the portable media device can receive a request from the accessoryto generate a remote GUI image for a third-party application. Therequest can also include specifications for the remote GUI imageincluding but not limited to type and amount of information items to beincluded in the remote GUI image. At block 1408, the portable mediadevice can execute the third-party application and generate the remoteGUI image based on the specification. Once generated, the remote GUIimage can be communicated to the accessory at block 1410. At block 1414,the portable media device can check whether it has received instructionsto exit the remote GUI mode. If yes, process 1400 can end at block 1414.If no, the portable media device can keep sending updated remote GUIimages or the video signal and process 1400 can return to block 1410.

It will be appreciated that process 1400 is illustrative and thatvariations and modifications are possible. Operations described assequential may be executed in parallel, order of operations may bevaried, and operations may be modified, combined, added or omitted. Forinstance, at block 1404, the accessory can also send information to theportable media device indicating whether the accessory supports a remoteGUI image. In some embodiments, the portable media device can start andstop sending remote GUI images based on explicit instructions from theaccessory, e.g., using commands EnterRemoteGUIMode andExitRemoteGUIMode, respectively. It is to be noted that actions inblocks 1406, 1408, and 1410 can occur whether or not the portable mediadevice is running a third-party application. For example, these actionscan also be performed for any native or first-party application that isresident on the portable media device.

In some embodiments, the portable media device may control the remoteGUI image of the third-party application based on the operatingenvironment of the accessory. For example, consider that the accessoryis an automobile audio/video head unit that is capable of receivinginformation from the portable media device. In this instance, if theautomobile is in motion, the accessory can send environment information,e.g., using the SetEnvInfo command, to the portable media device. Basedon this information, the portable media device can send a first remoteGUI image to the head unit for a media item that is currently beingplayed by the third-party application. The first remote GUI image caninclude only information that will not distract the driver. For example,the first remote GUI image may include only the name of the third-partyapplication or only title of the track being currently played. If theautomobile is subsequently put in a parked state with the media itemstill playing, the accessory can send the new environment information tothe portable media device. Based on the new environment information, theportable media player can send a second remote GUI image to theaccessory for the same media item. The second remote GUI image caninclude additional information about the media item and can also enableuser control inputs, e.g., buttons 116 a-h of FIG. 1.

In another embodiment, the accessory can dynamically change the remoteGUI image type during the operation, e.g., by sending a newSetPMDOptions command, rather than relying on the portable media deviceto make a decision based on environment information.

Thus, as described above, the portable media device can provide a remoteGUI image to the accessory for any third-party application resident onthe portable media device. The remote GUI image can include varyingamounts of information as specified by the accessory. In addition, ifthe operating system of the portable media device generates the remoteGUI image for multiple first-party and third-party applications, theremote GUI images for the different applications can have a standardlook and feel. This makes it easier for a user to read and interact withthe remote GUI image.

While the invention has been described with respect to specificembodiments, one skilled in the art will recognize that numerousmodifications are possible. For example, although embodiments describedabove may make specific reference to playback of media content, aportable media device may also provide other functionality such aspersonal information management, mobile telephony, and so on. In otherembodiments, any of this functionality can be made accessible through aremote GUI. Thus, for example, another remote GUI implementation mayprovide a hands-free telephone interface, e.g., for the driver of anautomobile. The driver can operate the remote GUI to answer or placetelephone calls, and the call can be conducted using a speaker andmicrophone controlled by the accessory, together with the mobile phoneconnection of the PMD. The remote GUI for telephony can resemble thePMD's native GUI for telephony, thus providing the user with a moreintuitive interface and potentially reducing distraction related tooperating an unfamiliar remote interface.

In addition, embodiments described above may make reference to aportable media device supplying the entire image that is displayed onthe accessory. In one alternative embodiment, the remote GUI imagesupplied by the portable media device can be displayed in a portion ofthe accessory's display screen while image data from other sources isdisplayed in other portions of the screen. For example, an accessory maydesignate a rectangular area within the display device for the remoteGUI. The portable media device can deliver pixel data (e.g., in digitalor analog form) for the remote GUI area of the display while theaccessory determines the pixels for the rest of the display. Forexample, the accessory can use a portion of the display for displayingnavigation information while another portion is used for the remote GUI.For portions of the display not in the remote GUI area, the portablemedia device can provide placeholder data (e.g., black pixels in ananalog video stream) or no data. (If the accessory has a touchscreen asa user input control, the accessory can selectively forward user inputsignals indicating a touched portion of the screen to the media playerbased on whether the touched portion is inside or outside the remote GUIarea.)

While the embodiments described above may make reference to specifichardware and software components, those skilled in the art willappreciate that different combinations of hardware and/or softwarecomponents may also be used and that particular operations described asbeing implemented in hardware might also be implemented in software orvice versa.

Computer programs incorporating various features of the presentinvention may be encoded on various computer readable media for storageand/or transmission; suitable media include magnetic disk or tape,optical storage media such as compact disk (CD) or DVD (digitalversatile disk), flash memory, and the like. Computer readable mediaencoded with the program code may be packaged with a compatible deviceor provided separately from other devices (e.g., via Internet download).

Thus, although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

What is claimed is:
 1. An electronic device, comprising: one or moreprocessors; a touchscreen display; one or more input devices; memory;and one or more programs, wherein the one or more programs are stored inthe memory and configured to be executed by the one or more processors,the one or more programs including instructions for: displaying, on thetouchscreen display having a designated area for displaying a seconduser-interface portion received from a remote device, a user interface,wherein the user interface includes a first user-interface portiongenerated by the electronic device and the second user-interface portiongenerated by the remote device, wherein the second user-interfaceportion is displayed in the designated area; detecting input from aninput device of the one or more input devices; and in response todetecting the input: determining whether the input is detected on thetouchscreen display within the designated area or outside of thedesignated area in accordance with a determination that the input isdetected on the touchscreen outside of the designated area,conditionally generating a response to the input based on a location ofthe input; and in accordance with a determination that the input isreceived on the touchscreen within the designated area: transmittinginformation indicative of the input to the remote device; aftertransmitting the information indicative of the input to the remotedevice, receiving, from the remote device, one or more graphicalelements; and responding to the input by updating the seconduser-interface portion to display the one or more graphical elementsreceived from the remote device, wherein the information indicative ofthe input transmitted to the remote device includes location informationindicating a location within the designated area of the touchscreendisplay at which a touch was detected.
 2. The device of claim 1, whereinthe information indicative of the input transmitted to the remote deviceincludes pixel coordinates of a location within the designated area ofthe touchscreen display.
 3. The device of claim 1, wherein theinformation indicative of the input transmitted to the remote deviceincludes information identifying a control with which an interactionoccurred.
 4. The device of claim 1, wherein the information indicativeof the input transmitted to the remote device includes informationindicating an operation associated with the input.
 5. The device ofclaim 1, wherein the transmitting of information indicative of the inputto the remote device occurs without interpretation by the remote device.6. The device of claim 1, wherein, in response to receiving informationindicative of the input transmitted to the remote device, the remotedevice is configured to perform at least one action.
 7. The device ofclaim 6, wherein the at least one action performed by the remote deviceincludes at least one of: updating a user interface of the remotedevice, providing instructions to the electronic device to update a userinterface of the electronic device, initiating playback of a media file;pausing playback of a media file, modifying a setting of the remotedevice; and sending instructions to the electronic device to modify asetting of the electronic device.
 8. The device of claim 2, wherein theremote device is configured to interpret the pixel coordinates of thelocation within the designated area of the touchscreen display andperform at least one action based on the interpretation of the pixelcoordinates.
 9. The device of claim 2, wherein the one or more graphicalelements are selected by the remote device in accordance with the pixelcoordinates of the location within the designated area of thetouchscreen display.
 10. The device of claim 1, wherein the one or moregraphical elements are selected by the remote device in accordance withthe location information indicating the location within the designatedarea of the touchscreen display at which a touch was detected.
 11. Thedevice of claim 1, wherein the one or more graphical elements aregenerated by the remote device without any intervention by theelectronic device.
 12. The device of claim 1, wherein at least one ofthe one or more graphical elements indicate a functionality of theremote device that was invoked in response to interpretation of theinput by the remote device.
 13. The device of claim 1, wherein theelectronic device is an in-vehicle media control unit and the remotedevice is a portable media device.
 14. A method comprising: at anelectronic device with one or more processors, a touchscreen display,one or more input devices, memory: displaying, on the touchscreendisplay having a designated area for displaying a second user-interfaceportion received from a remote device, a user interface, wherein theuser interface includes a first user-interface portion generated by theelectronic device and the second user-interface portion generated by theremote device, wherein the second user-interface portion is displayed inthe designated area; detecting input from an input device of the one ormore input devices; and in response to detecting the input: determiningwhether the input is detected on the touchscreen display within thedesignated area or outside of the designated area; in accordance with adetermination that the input is detected on the touchscreen outside ofthe designated area, conditionally generating a response to the inputbased on a location of the input; and in accordance with a determinationthat the input is received on the touchscreen within the designatedarea: transmitting information indicative of the input to the remotedevice; after transmitting the information indicative of the input tothe remote device, receiving, from the remote device, one or moregraphical elements; and responding to the input by updating the seconduser-interface portion to display the one or more graphical elementsreceived from the remote device, wherein the information indicative ofthe input transmitted to the remote device includes location informationindicating a location within the designated area of the touchscreendisplay at which a touch was detected.
 15. The method of claim 14,wherein the information indicative of the input transmitted to theremote device includes pixel coordinates of a location within thedesignated area of the touchscreen display.
 16. The method of claim 14,wherein the information indicative of the input transmitted to theremote device includes information identifying a control with which aninteraction occurred.
 17. The method of claim 14, wherein theinformation indicative of the input transmitted to the remote deviceincludes information indicating an operation associated with the input.18. The method of claim 14, wherein the transmitting of informationindicative of the input to the remote device occurs withoutinterpretation by the remote device.
 19. The method of claim 14,wherein, in response to receiving information indicative of the inputtransmitted to the remote device, the remote device is configured toperform at least one action.
 20. The method of claim 19, wherein the atleast one action performed by the remote device includes at least oneof: updating a user interface of the remote device, providinginstructions to the electronic device to update a user interface of theelectronic device, initiating playback of a media file; pausing playbackof a media file, modifying a setting of the remote device; and sendinginstructions to the electronic device to modify a setting of theelectronic device.
 21. The method of claim 15, wherein the remote deviceis configured to interpret the pixel coordinates of the location withinthe designated area of the touchscreen display and perform at least oneaction based on the interpretation of the pixel coordinates.
 22. Themethod of claim 15, wherein the one or more graphical elements areselected by the remote device in accordance with the pixel coordinatesof the location within the designated area of the touchscreen display.23. The method of claim 14, wherein the one or more graphical elementsare selected by the remote device in accordance with the locationinformation indicating the location within the designated area of thetouchscreen display at which a touch was detected.
 24. The method ofclaim 14, wherein the one or more graphical elements are generated bythe remote device without any intervention by the electronic device. 25.The method of claim 14, wherein at least one of the one or moregraphical elements indicate a functionality of the remote device thatwas invoked in response to interpretation of the input by the remotedevice.
 26. The method of claim 14, wherein the electronic device is anin-vehicle media control unit and the remote device is a portable mediadevice.
 27. A non-transitory computer readable storage medium storingone or more programs, the one or more programs comprising instructions,which when executed by an electronic device with one or more processors,a touchscreen display, one or more input devices, and memory, cause thedevice to: display, on the touchscreen display having a designated areafor displaying a second user-interface portion received from a remotedevice, a user interface, wherein the user interface includes a firstuser-interface portion generated by the device and the seconduser-interface portion generated by the remote device; detect input froman input device of the one or more input devices; and in response todetecting the input: determine whether the input is detected on thetouchscreen display within the designated area or outside of thedesignated area; in accordance with a determination that the input isdetected on the touchscreen outside of the designated area,conditionally generate a response to the input based on a location ofthe input; and in accordance with a determination that the input isreceived on the touchscreen within the designated area: transmitinformation indicative of the input to the remote device; aftertransmitting the information indicative of the input to the remotedevice, receiving, from the remote device, one or more graphicalelements; and responding to the input by updating the seconduser-interface portion to display the one or more graphical elementsreceived from the remote device, wherein the information indicative ofthe input transmitted to the remote device includes location informationindicating a location within the designated area of the touchscreendisplay at which a touch was detected.
 28. The non-transitory computerreadable storage medium of claim 27, wherein the information indicativeof the input transmitted to the remote device includes pixel coordinatesof a location within the designated area of the touchscreen display. 29.The non-transitory computer readable storage medium of claim 27, whereinthe information indicative of the input transmitted to the remote deviceincludes information identifying a control with which an interactionoccurred.
 30. The non-transitory computer readable storage medium ofclaim 27, wherein the information indicative of the input transmitted tothe remote device includes information indicating an operationassociated with the input.
 31. The non-transitory computer readablestorage medium of claim 27, wherein the transmitting of informationindicative of the input to the remote device occurs withoutinterpretation by the remote device.
 32. The non-transitory computerreadable storage medium of claim 27, wherein, in response to receivinginformation indicative of the input transmitted to the remote device,the remote device is configured to perform at least one action.
 33. Thenon-transitory computer readable storage medium of claim 32, wherein theat least one action performed by the remote device includes at least oneof: updating a user interface of the remote device, providinginstructions to the electronic device to update a user interface of theelectronic device, initiating playback of a media file; pausing playbackof a media file, modifying a setting of the remote device; and sendinginstructions to the electronic device to modify a setting of theelectronic device.
 34. The non-transitory computer readable storagemedium of claim 28, wherein the remote device is configured to interpretthe pixel coordinates of the location within the designated area of thetouchscreen display and perform at least one action based on theinterpretation of the pixel coordinates.
 35. The non-transitory computerreadable storage medium of claim 28, wherein the one or more graphicalelements are selected by the remote device in accordance with the pixelcoordinates of the location within the designated area of thetouchscreen display.
 36. The non-transitory computer readable storagemedium of claim 27, wherein the one or more graphical elements areselected by the remote device in accordance with the locationinformation indicating the location within the designated area of thetouchscreen display at which a touch was detected.
 37. Thenon-transitory computer readable storage medium of claim 27, wherein theone or more graphical elements are generated by the remote devicewithout any intervention by the electronic device.
 38. Thenon-transitory computer readable storage medium of claim 27, wherein atleast one of the one or more graphical elements indicate a functionalityof the remote device that was invoked in response to interpretation ofthe input by the remote device.
 39. The non-transitory computer readablestorage medium of claim 27, wherein the electronic device is anin-vehicle media control unit and the remote device is a portable mediadevice.
 40. An electronic device, comprising: one or more processors;memory; and one or more programs, wherein the one or more programs arestored in the memory and configured to be executed by the one or moreprocessors, the one or more programs including instructions for: whilethe device is coupled to an accessory that includes a touchscreendisplay and an input device, receiving a request to provide a userinterface for display at a designated area of the touchscreen display ofthe accessory; and in response to receiving the request: generating afirst user-interface portion for display in the designated area of thetouchscreen display; and initiating display of the first user-interfaceportion in the designated area of the touchscreen display of theaccessory, wherein the first user-interface portion is displayed in thedesignated area of the touchscreen display while a second user-interfaceportion generated by the accessory is concurrently displayed in a secondregion of the touchscreen display that is distinct from the designatedarea of the touchscreen display; and after initiating display of thefirst user-interface portion in the designated area of the touchscreendisplay of the accessory, receiving input that is indicative of inputreceived on the touchscreen within the designated area; and in responseto receiving the input: generating an updated first user-interfaceportion including one or more graphical elements in accordance with theinput; and initiating display of the updated first user-interfaceportion in the designated area of the touchscreen display, wherein theinformation indicative of the input received on the touchscreen includeslocation information indicating a location within the designated area ofthe touchscreen display at which a touch was detected.
 41. The device ofclaim 40, wherein the one or more programs include instructions fortransmitting information identifying a portion of the displaycorresponding to the designated area of the touchscreen display.
 42. Thedevice of claim 40, wherein generating a first user-interface portionincludes generating placeholder data for the second user-interfaceportion.
 43. The device of claim 40, wherein the request includes atleast one of configuration, environment, and status information.
 44. Thedevice of claim 40, wherein the first user-interface portion is adaptedto the configuration of the first region of the display.
 45. The deviceof claim 40, wherein the one or more programs include instructions for:after receiving the request, transmitting a list of remote userinterface image types; receiving a selection of a user interface imagetype from the list of remote user interface image types; and generatingthe first user-interface portion using the selected user interface imagetype.
 46. The device of claim 40, wherein the one or more programsinclude instructions for: receiving a request to stop providing the userinterface for display at the accessory; and; in response to receivingthe request to stop providing the user interface for display at theaccessory; stop generating the updated first user-interface portion.